Platform Comuting Test Dev Survey Results FINAL

Research Report on Virtualization and Cloud Computing in Test/Dev Environments
Platform Computing
www.platform.com
Copyright 2009 Platform Computing Corporation. All Rights Reserved.
Background: Methodology
Objective: Examine the challenges of managing Test/Dev environments and the opportunity for server virtualization and cloud computing to address them Fielded: Jun-Aug 2009 Method: Telephone surveys Sample: IT architecture and operations managers responsible for development and test infrastructure (N=83), at North American firms, cross-industry.
SMB: n=57 (<500 Test/Dev servers) large enterprise: n=25 (>500 Test/Dev servers)
Fielding and data analysis support by third party vendors

Fielding: Primary Point (http://www.primarypoint.com/) Survey development and analysis support: Taneja Group (http://www.tanejagroup.com/)
2 Copyright 2009 Platform Computing Corporation. All Rights Reserved.
Key Findings: The Case for Private Cloud

1. Significant investment is being allocated to solve Test/Dev operational and budgetary challenges: 50% of respondents will have a funded project start in 2009, while 66% of respondents will have a funded project started by end of 2010 Application teams are using shared infrastructure in production today, supporting the theory that Cloud Computing is more about a new business model for IT delivery than new technology Few firms have adopted a hosted environment outside the firewall. This helps to explain why many firms are starting with private cloud; get the business model right before introducing additional complexity involved with transitioning to external infrastructure The top challenge identified in managing Test/Dev environments: the need to manage virtual and physical resources separately Experienced users confirmed that server virtualization is an enabler, but adds several layers of control and cost issues that must be addressed by sharing, process, workflow, and other management capabilities
2.
3.
4.
5.
Industry Breakdown
Industry Breakdown: ALL
Pharma Other 2% 1% Health 8% Banking 20%
Industry Breakdown: LARGE (>500)

Other 4% Banking 32% S/W 28% Gov 8% Manuf 16%
S/W 18% Telco 14%
Gov 18% Manuf 17%
Telco 12%
Takeaway: Broad industry representation. Banking and Software segments dominate the Large category with a combined 60% of respondents.
Size of Test/Dev Environment

Number of Physical Test/Dev Servers Distribution
Large > 500 31%
40 35 30 25 20 15 10 5 0
SMB < 500 69%
Takeaway: Broad coverage of infrastructure scale. Banking accounts for 40% of the environments that are greater than 1000 servers.
Prevalence of Virtualization in Test/Dev

Does your company use virtualization in Test/Dev environments?
No 13% Yes 89%
If Yes: Approximately what percentage of physical hosts are supporting virtualization?

0-25% 25-50%
39 14 8 9
53% of respondents have virtualized less than 25% of their physical servers
50-75%
75-100% Dont know/Refused
Takeaway: While virtualization is used heavily in Test/Dev environments, there remains a significant amount of work being done on non-virtualized infrastructure. Server virtualization prevalence matches industry estimates of 20-30% market penetration.
Hypervisor Vendor Prevalence in Test/Dev

What hypervisor vendors are you using?
VMware 80
Average Number of VMs per virtual host?
1-10
20
Microsoft
18
65% Homogeneous 35% Heterogeneous
Average: 20
11-20
Citrix
15
Other
16
> 20
13
Takeaway: VMware prevalence is 95%, but 35% run at least one other hypervisor. Consolidation ratios: 20/host on average is high (with respect to production server virtualization consolidation ratios), indicating dense, highly shared environments and significant contention for limited, shared physical server resources.
Resource Sharing
Objectives: Determine the types of resources currently shared Determine use cases for shared resources Explore prevalence of internal (private cloud) sharing and external (hosted, outsourced, public cloud) sharing Establish current views of limitations and challenges in Test/Dev for both internal shared and external hosting
Resource Sharing in Test/Dev

Does your company share servers in Test/Dev environments? No 8% If Yes, do you share servers between the SDLC phases (Dev, Test, UAT)?
No 14%
Yes 92%
Yes 86%
Takeaway: Development teams have a high degree of comfort using shared infrastructure. Of the small group that dont (8%), most plan to within 2 years. Teams share across nearly all phases (86% share across at least Dev, Unit Test, Integration Test & Performance Test phases).
Resource Sharing for Other Activities

Do you currently, or plan to, use your Test/Dev environment for other activities?
If Yes, What activities?
Training
53
No 31%
Yes 69%
Demos, Prototyping
47
Disaster Recovery
Takeaway: Sharing is prevalent, even for non-Test/Dev activities. Disaster Recovery is likely an emerging use case, based on separate Taneja Group research.
10
Use of Outsourced Infrastructure

Do you use off-premise or hosted servers for Test/Dev (outsourcing)? If No: Why Not?
Lack of Control Immature Technology We Don't Have the Need
24 19 13 10 3 3 2
Yes 18% No 82%
Lack of Security Vendors are Too Small Too Expensive No Time
Takeaway: Few firms have adopted a hosted environment outside the firewall. Lack of Control and Immature Technology are the top functional blockers, but a significant number have not been effectively sold on benefits (We Dont Have Need)
Use of Outsourced Infrastructure

Do you use off-premise or hosted servers for Test/Dev (outsourcing)? If Yes: What percentage of your Test/Dev servers are currently outsourced?
No 82%
Yes 18%
Less Than One Quarter
More Than One Quarter
71%
29%
Takeaway: Of those who have outsourced, 71% have outsourced less than 25% of their Test/Dev servers. Only a few vendors were named: Amazon (3), Savvis (2), Skytap (1), Surgient (1), ADP (1)
Sharing Resources in Production

Are you using shared infrastructure or a private cloud for Production enterprise applications? If Yes: Do Production applications share the same or different resource pools as Test/Dev?
No 24%
Yes 76%
Different Pools
Same Pools
30%
70%
Takeaway: Application teams are using shared infrastructure in production today, supporting the theory that Cloud Computing is more about a new business model for IT delivery than new technology. 30% of shared pools host both Test/Dev and Production applications, indicating a rising comfort level with sharing overall. Taneja Group experience confirms emergence of Test/Dev/Prod sharing.
Shared Production Applications

If Yes, what would you say are your top three Production applications using shared infrastructure or private cloud?
Top 3 Shared Production Applications
CRM
21 20 19 15
Finance
Web
ERP
Intranet HR Business Critical System Software Web 2.0 2 3 8
13 8
Takeaway: Supporting the virtualization trend, enterprise applications that are within the firewall continue to be the primary production applications that are sharing resources. Business critical applications are at the early stage of moving to a shared environment.
Sharing Resources in Production

Are you using shared infrastructure or a private cloud for Production enterprise applications? If No, do you know if the company has any plans to implement shared or private cloud for Production?
70% of those with plans to share in Production will do so in the next 12 months
Yes 76%
No 24%
No 45%
Yes 55%
Takeaway: Of the 24% who are not sharing, 70% have plans to do so in the next 12 months (70%, N=7/10). Top apps planned to share resources: ERP, Finance, CRM, Web
Types of Applications Being Developed

Across your enterprise Test/Dev environments, what kinds of applications are being developed? Web 50 45 42 38
Intranet
Finance
Business Critical
System Software Web 2.0 CRM HR ERP
35
34
30
30 28
Takeaway: Broad mix, focused on web/intranet, finance, and business critical (specific to the industry/business). Finance and Web apps are also in the top 3 types of apps currently shared in production.
Test/Dev Challenges
Objectives Determine the types of user and IT support challenges Focus on server & virtual machine lifecycle management issues (vs. SDLC management) Explore current projects, future planning, and funding to address Test/Dev challenges
Test/Dev Operations Challenges

Please rank the impact of these operational challenges in your Test/Dev environment:
Med/High Impact
IT manages virtual and physical resources separately Sharing resources among teams is difficult IT does too much configuration work Users complain about lack of control Users wait too long for resources Users are not returning machines to the pool Users do too much configuration work
No/Low Impact
55% 47% 46% 45% 42% 36%
45% 53% 54% 55% 58% 64%
33%
67%
Takeaway: Top challenge: the need to manage virtual and physical resources separately. The top three challenges relate to lack of cross-domain management, server lifecycle management, and lab automation.
Existing Projects to Address Challenges

Do you currently have any projects around any of these challenges? If Yes: Which ones? (Top Five)
IT spends too much time on configuration Users wait too long for resources
13 12
No 59%
Yes 41%
Users spend too much time on configuration
11 11 8
Managing virtual and physical separately

Sharing environments is difficult
Takeaway: Top projects focus on configuration (both IT and user configuration efforts), the need for self-service (wait too long), and better environment sharing across the software development lifecycle.
Planned Projects to Address Challenges

If No, do you currently have any planning around these challenges?
If Yes: Which ones? (Top Five)

IT spends too much time on configuration Users spend too much time on configuration
6
6 5
Users wait too long for resources
No 67%
Yes 33%
Sharing environments is difficult
5
Managing virtual and physical separately
Takeaway: Delivering good configurations to Test/Dev teams quickly, and allowing them to recreate and share them with a minimum of configuration, are the focus of most current and planned projects.
Operations Impact: Time Lost

87% Claim User Time Is Lost Due To Ops Challenges. How Much? 89% Claim IT Support Time Is Lost Due To Ops Challenges. How Much?
19% say more than 25% of time is lost
>25% 8% > 10% < 25% 42% <10% 50%
>25% 19% > 10% < 25% 30% <10% 51%
Takeaway: Less time is lost by Test/Dev users than by IT support, but for both the loss is significant (50% agree that more than 10% of both user and admin time is lost due to challenges).
Test/Dev Budget Challenges

Please rank the impact of these budgetary challenges in your Test/Dev environment?
Med/High Impact
No budget to add more resources
No/Low Impact
59%
41%
22% Have Projects Underway Around Budgetary Challenges
Struggle to track utilization
51%
49%
Unable to charge BU's for what they use
51%
49%
Takeaway: Significant recession effect (highest impact of all challenges) is likely retarding the market for Test/Dev solutions more than any other factor in 2009. Many respondents indicated they would have explored a solution except for time/budget.
22
Cost Analysis of Lost Time

Data:
Dev Team: 87% agree time is lost
Of these, 50% say <10% lost, 42% say 1025%, and 8% say > 25% Of these, 51% say <10% lost, 30% say 1025%, and 19% say > 25%
Assumptions:
Fully-burdened developer at $150K per year
Implication: 50% of respondents lose more than $15K per developer per year Implication: 30% of respondents lose more than $10K per admin per year
IT Team: 89% agree time is lost
Fully-burdened admin at $100K per year
Cost Analysis:
For an environment with 100 developers/testers, with 5 admins 50% will lose at least $1.5M per year in developer time 49% will lose at least $50K per year in IT admin time
Developer/Admin Ratio = 10/1 (conservative)

49% of IT staff distribution is in application development and support , and 14% of IT staff distribution is in the data center *
Takeaway: Lost developer/tester time is a much bigger challenge and cost issue than lost IT admin time
* Source: Gartner IT Spending and Staffing Report, 2009
The Solution Opportunity

Objectives Determine familiarity and impressions of current on-premise and hosted lab automation and lifecycle management solutions Explore existing projects and planning around solution evaluation Estimate budget size
Use of Test/Dev Provisioning Tools

Which provisioning tools are you using?
61% Have Homegrown Tools or No Tools. Significant Lack of Investment to Date in Test/Dev Provisioning Software
Other/ Homegrown 27%

VMware 11% BMC 11%
None 34%
HP/ Opsware 17%
Takeaway: There is a clear lack of vendor penetration for automated provisioning tools in Test/Dev environments, which explains the high level of reported challenges and the lack of experience with lifecycle or lab automation (higher-order mgmt tools).
Effectiveness of Virtualization
Will server virtualization on its own address the most important challenges in your Test/Dev environment?
Representative Why Not? Answers:
Doesn't take in to account integration and interoperability issues these are the real challenges. Need a good service-based orientation managing business services not hardware.
Yes 28%
No 72%
You still need process automation; virtualization is just an enabling technology.

Our biggest challenges in Test/Dev are the same whether the servers are virtualized or not. Our biggest problem now is getting the configuration[s] right.
Takeaway: Virtualization on its own was expected to solve the challenges only by those respondents new to using the technology. The majority of experienced users confirmed that virtualization is an enabler, but adds several layers of control and cost issues that must be addressed by process, workflow, sharing and other management capabilities.
Existing Solution Experience

Have you evaluated or tested any lab automation or virtualization lifecycle management software?
VMware Surgient
Which ones?
27 4 2 2 2
No 69%
40% have not evaluated due to Lack of Time or Budget
Yes 31%
VMLogix Microsoft Enigmatec
Takeaway: VMware Lab Manager dwarfs other solutions, but budgetary issues have stalled a large percentage of evaluations. Few respondents indicated functional problems with lab management solutions.
Funding to Address Challenges

Do you have a currently funded project to build or purchase Test/Dev shared infrastructure solutions?
If Yes: can you tell us the approximate budget amount?

$250,000 + 100,000-$250,000
No 58%
Yes 42%
50,000-$100,000 0-$50,000 Don't Know/Refused 0 5 10 15 20
Takeaway: Significant investment is being allocated to solve Test/Dev operational and budgetary challenges.
Funding to Address Challenges

Do you have a currently funded project to build or purchase Test/Dev shared infrastructure solutions?
If No, do you have one planned to start this year or next year?
Yes 42%
No 58%
Yes 38%
No 62%
Takeaway: 38% of respondents that are not investing to solve Test/Dev challenges today plan to do so in the next two years. 50% of respondents will have a funded project start in 2009, while 66% of respondents will have a funded project started by end of 2010.
Additional Industry Segment Analysis
30
Industry Segment Consistency

Virtualization Adoption
Size of environments (Banking outlier) Number of physical hosts supporting virtualization Use of virtualization (Manuf outlier) Hypervisors used Use of shared servers (SW and Health outlier) Sharing between stages of SDLC Using shared servers in production (Telco outlier) Use Test/Dev for other non-prod work, i.e. demos (Gov and Health outliers) Application types developed Number that have evaluated lab automation (SW and Health outliers) Number that have current funding (Banking outlier) Planned funding for this year
Sharing of Resources
Solution Opportunity
31
Industry Segment Outliers

Banking
Heavier outsourcing: 31% (vs. 18% avg.) Little sharing between Test/Dev and Production: 11% (vs. 30% avg.) Large: accounts for 40% of the environments that are > 1000 hosts High projects around operational challenges: 69% (vs. 41% avg.) 15% believe virtualization is enough to solve problems (vs. 28% avg.) 62% have current funding (vs. 42% avg.)
Government
High sharing between Test/Dev and Prod: 56% (vs. 30% avg.) High use of Test/Dev for other non-prod work, i.e. demos: 91% (vs. 69% avg.) Low projects around operational challenges: 18% (vs. 41% avg.) More believe virtualization is enough to solve problems: 55% (vs. 28% avg.) High number with planned funding for next year: 50% (vs. 27% avg.)
32
Industry Segment Outliers (cont.)

Manufacturing
Only 64% use virtualization (vs. 90% avg.) No outsourcing: 0% (vs. 18% avg.) No sharing between Test/Dev and Prod: 0% (vs. 30% avg.) High number with planned funding for next year: 71% (vs. 27% avg.)
Software
Only 70% share servers (vs. 92% avg.) Lighter outsourcing: 10% (vs. 18% avg.) No sharing between Test/Dev and Prod: 0% (vs. 30% avg.) Low projects around operational challenges: 20% (vs. 41% avg.) High number have tested lab automation: 50% (vs. 31% avg.)
33
Industry Segment Outliers (cont.)

Telecommunications
Heavier outsourcing: 36% (vs. 18% avg.) Lighter shared servers in production: 55% (vs. 76% avg.) High sharing between Test/Dev and Prod: 83% (vs. 30% avg.) Low projects around operational challenges: 9% (vs. 41% avg.) High number with planned funding for next year: 50% (vs. 27% avg.)
Health Care
Only 57% share servers (vs. 92% avg.) No outsourcing: 0% (vs. 18% avg.) No sharing between Test/Dev and Prod: 0% (vs. 30% avg.) Low use of Test/Dev for other non-prod work, i.e. demos: 29% (vs. 69% avg.) High number have tested lab automation: 71% (vs. 31% avg.) No planned funding for next year: 0% (vs. 27% avg.)
34
For more information

Contact Martin Harris, Director Product Management, mharris@platform.com Or visit http://www.platform.com/company/contact-us
35

Platform Comuting Test Dev Survey Results FINAL

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Platform Comuting Test Dev Survey Results FINAL

Uploaded by

Copyright:

Available Formats

Research Report on Virtualization and Cloud Computing in Test/Dev Environments

Copyright 2009 Platform Computing Corporation. All Rights Reserved.

Fielding and data analysis support by third party vendors

Key Findings: The Case for Private Cloud

Industry Breakdown: LARGE (>500)

S/W 18% Telco 14%

Gov 18% Manuf 17%

Size of Test/Dev Environment

SMB < 500 69%

Prevalence of Virtualization in Test/Dev

If Yes: Approximately what percentage of physical hosts are supporting virtualization?

Hypervisor Vendor Prevalence in Test/Dev

Average Number of VMs per virtual host?

65% Homogeneous 35% Heterogeneous

Copyright 2009 Platform Computing Corporation. All Rights Reserved.

Resource Sharing in Test/Dev

Resource Sharing for Other Activities

Copyright 2009 Platform Computing Corporation. All Rights Reserved.

Use of Outsourced Infrastructure

Yes 18% No 82%

Lack of Security Vendors are Too Small Too Expensive No Time

Use of Outsourced Infrastructure

Less Than One Quarter

More Than One Quarter

Sharing Resources in Production

Shared Production Applications

Sharing Resources in Production

Types of Applications Being Developed

Copyright 2009 Platform Computing Corporation. All Rights Reserved.

Test/Dev Operations Challenges

Existing Projects to Address Challenges

Users spend too much time on configuration

Managing virtual and physical separately

Planned Projects to Address Challenges

If Yes: Which ones? (Top Five)

Users wait too long for resources

Operations Impact: Time Lost

>25% 8% > 10% < 25% 42% <10% 50%

>25% 19% > 10% < 25% 30% <10% 51%

Test/Dev Budget Challenges

22% Have Projects Underway Around Budgetary Challenges

Struggle to track utilization

Unable to charge BU's for what they use

Copyright 2009 Platform Computing Corporation. All Rights Reserved.

Cost Analysis of Lost Time

IT Team: 89% agree time is lost

Fully-burdened admin at $100K per year

Developer/Admin Ratio = 10/1 (conservative)

The Solution Opportunity

Copyright 2009 Platform Computing Corporation. All Rights Reserved.

Use of Test/Dev Provisioning Tools

Other/ Homegrown 27%

HP/ Opsware 17%

Representative Why Not? Answers:

You still need process automation; virtualization is just an enabling technology.

Existing Solution Experience

VMLogix Microsoft Enigmatec

Funding to Address Challenges

If Yes: can you tell us the approximate budget amount?

50,000-$100,000 0-$50,000 Don't Know/Refused 0 5 10 15 20

Funding to Address Challenges

Additional Industry Segment Analysis

Copyright 2009 Platform Computing Corporation. All Rights Reserved.