Converging network with HP FlexFabric and FlexManagement

This week, we shine the spotlight on network convergence and management in the FlexNetwork portfolio, specifically the HP Virtual Connect FlexFabric modules and the Intelligent Management Control (IMC) software.  Networking is generally not my forte, so some of this is a stretch for me, but I use and understand Virtual Connect. FlexFabric is an implementation of Virtual Connect while IMC has been branded as FlexManagement in the portfolio and is used to encompass management for all of the FlexNetwork portfolio.

Some Basics on Virtual Connect

FlexFabric is a particular type of Virtual Connect module and fits into the FlexNetwork portfolio.  Virtual Connect attempts to address many problems, but primarily it is about reducing the amount of physical wiring and switch ports required to cable a blade system, about reducing human-caused errors due to complex cabling, and about adding the ability to pre-wire the entire enclosure for life and pre-allocate all Ethernet and SAN requirements during first install.

The major selling point of Virtual Connect is the ability to virtualize the network MAC addresses and the Fiber Channel worldwide names/ID (WWID).  Virtual Connect allows for server profiles to be built and assigned to blade hardware and then provides the ability to move the profile from physical blade to physical blade without the need to reconfiguration.  The virtualized WWID and MAC addresses are used instead of the physically assigned addresses that are provided from the manufacturer.

When first introduced, Virtual Connect (VC) existed as separate Ethernet and Fiber Channel modules.  Installed in pairs, each VC module is physically “wired” through a mid-plane to specific blade server ports.  Both the Ethernet and Fiber Channel pairs share a 10Gb “cross connect” connection on the backplane which allows each pair of modules to talk between each other and pass traffic to uplinks from each blade port, even if the uplink is on the other module.  The cross connects also allow for rerouting traffic to the other interconnect bay should an upstream network switch lose connection or become isolated.

Incremental Advancements

The first enhancement for Virtual Connect came with the introduction of Flex-10 technology in 2009, which took a 10Gb network interface on the motherboard and split it into 4 FlexNICs which are LAN on Motherboard (LOM) interfaces and allows the ability to set a fixed amount of bandwidth per FlexNIC.  Each interface is presented to the operating system on the blade as a separate NIC.

Covered in the call last week, the newest innovation in Virtual Connect technology is the ability to condense both Fiber Channel and Ethernet all onto a single set of interconnect modules, known as FlexFabric modules.  FlexFabric allow for the FlexNICs to present either 3 NICs and 1 FCoE (Fiber Channel over Ethernet) converged port to a server or 4 NICs (like the previous Flex-10), depending on server need.  The FlexFabric module removes the need for separate Fiber Channel adapters in a mezzanine slot on a blade and instead uses a Fiber Channel over Ethernet converged LOM.  From the FlexFabric Interconnect module, Fiber Channel uplinks are sent to the Fiber Channel switches and the traditional Ethernet uplinks are sent to network switches.  As the name implies, the ports are flexible and each of the SPF ports is capable of running Ethernet or Fiber Channel uplinks.

With any new technology, practices for security and monitoring must change and adapt to the innovation.  In general, security integration appears to be an afterthought in many cutting edge enhancements, taking VMware for example.  VMware is the most common example when we think of virtualization technology, but when virtual switching was first implemented, there was no way to view inter-VM traffic that never left the host.   In many cases, administrators were faced with a black box making it impossible to monitor, inspect or halt malicious traffic.

With Virtual Connect, some of the same issues apply, but HP has offered at least one solution to administrators and security officers.  HP’s Virtual Connect technology does allow for a network mirror port which can replicate all traffic out for inspection, so even traffic that never leaves the enclosure thanks to the way Virtual Connect implements can be inspected and send alarms accordingly.  It may not be as good as an in-line security solution that can actively block malicious traffic, but at least administrators can gain visibility.

A More Intelligent Way to Manage Network Infrastructure

As part of our call last week, we were also shown HP’s Intelligent Management Center, or IMC.  This software is a control and monitoring software for heterogenous switches and routers in the datacenter all from a single, common interface.  IMC is an impressive offering from HP, which gives network administrators a single interface to learn to provision all of their switches.  With a hardware compatibility list of over 5,000 devices, the IMC is a capable platform to control both your HP, Cisco and other vendor’s network gear.

IMC addresses the problem of swivel chair management where administrators must monitor multiple, vendor-provided management products for each different vendor or product line represented in your datacenter.  But even for a company that has standardized on non-HP network gear, IMC is a powerful interface that can be put to work for them for more than just monitoring and management.

IMC is a modules that can be added in for user access management which can centralize user accounts with a full-featured RADIUS server that can be used 802.1X, VPN, and wireless authentication.  In addition the UAM module adds features to lock down and secure corporate devices by preventing IP and user account spoofing and prevent address conflicts.  The same package can also be used to lock down corporate PC’s to prevent use of USB and external storage devices.  Another module that plugs into IMC is Endpoint Admission Defense tools, which can be used to policy control clients and ensure that devices on the network are safe for the network, patched and have up to date antivirus definitions.

IMC provides a great view into virtualized networking (as in VMware) and can monitor the virtualized networking to a very granular level. It exposes what has been a black box of virtual networking using vendor provided API’s.    At present, IMC support VMware and Hyper-V but will grow to include XenServer and KVM in 2012, if things go as planned.


All in all, the entire series for the Blogger Reality Show has focused on ways to converge infrastructure for simplicity and ease of management.  Each of the HP offerings approaches convergence on different sections of the IT puzzle. In HP’s product line, we have seen the basic building blocks of convergence with servers and the Bladesystem, we have seen converged storage solutions built on x86 hardware, we have seen these solutions built into larger solutions for virtualization and cloud, and finally we have investigated the solutions HP has to converge and management networking.

The Reality Show has been a very cool thing to take part of.  From a blogging perspective, it has been very cool to get judges feedback and to learn and stretch myself by trying new ways to promote the blog and posts.  The winner will be named next week at VMworld and I’ll try to post an expanded post about the contest after we wrap next week.  So, now it is your turn again…  Vote and comment.

This is the third and final post for Thomas Jones’ Blogger Reality Show sponsored by HP and Ivy Worldwide. I ask that readers be as engaged and responsive as possible during this contest.  I would like to see comments and conversations that these entries spark, tweets and retweets if it interests you and I also request that you vote for this entry using the thumbs up/thumbs at the top of this page.  As I said earlier, our readers play a large part in scoring, so participate in my blog and all the others!

Simplifying IT support and deployments with converged systems

All IT solutions will experience problems at some point in their life.  Supporting IT solutions is difficult, time-consuming and costly, but also a fact of life – a fact as a systems administrator I am thankful for.  It means, I have a job.  Problem solving skills are absolutely necessary, but all administrators need the expert help of vendors’ support departments when our knowledge runs into something we just don’t know.

Unfortunately, when multiple vendors’ products are coupled together as a solution, support can become nasty as vendors point back and forth at each other while trying to get to a resolution.  The more complex the solution, for instance a SAN, the more difficult to troubleshoot through the multiple layers of software, firmware and hardware, even multiple vendors of the solution.  And, I believe, the hassle has made customers seek a better way.

Finding a better way

In my employer’s case, they chose to standardize with a single vendor long before I joined the staff.   We have stuck with servers and storage hardware from the single vendor, including their certified part upgrades (no third party upgrade components).  We chose to do this to simplify our support and avoid finger-pointing.

The vendor we standardized with was HP, and the reason was that they offered an entire line of products under their umbrella to meet our needs.  By the time I joined the staff in 2006, we were already HP heavy, except where a specific Unix was required by another vendor.   What we wanted as a customer was the quickest and easiest route  to a resolution, with the least resistance and finger-pointing, when a problem came up.  Even beyond the hardware solutions, HP has handled our software support for Microsoft, RedHat and VMware for many years.  We wanted this because the software companies could not finger point at the hardware or vice versa – HP was doing it all.  Sure, it might happen between teams in HP occasionally, but we could easily escalate our case and have a manager bring this to a resolution.  It has worked well for our needs.

Having all this expertise in-house is an advantage that HP is now branding under the name “Converged Systems” or the “Instant-On Enterprise”.  Earlier this week, I attended a webinar for the Blogger Reality Contest where HP unpacked more of its converged solutions strategies.  HP is bringing together all of the pieces spread throughout its portfolio into specialized solutions.  Its not a new concept, in my opinion, but one that some customers have been already using for years on their own.  HP has improved on this by tweaking configurations  to squeeze performance out of configurations and adding software to ease installation and management of the solutions.

Building Upwards – HP VirtualSystem

HP introduced VirtualSystem in June as a modular, easy and quick way to implement virtualization in customer datacenters.  The VirtualSystem solution is a full package of storage and compute resources plus the software tools to quickly and easily deploy a virtual stack in an environment.

For HP VirtualSystem, the key benefits are:

  • Quick built out timeframe
  • Automation through Insight Control suite components
  • Monitoring through the Insight Dynamics suite components
  • Improved virtual machine performance, cost and scale due to purpose built hardware
  • Ability to upgrade to CloudSystem for fully automated IT
  • Single point of contact for support – HP for compute, storage and software, including hypervisor

HP VirtualSystem comes in 3 levels (shown below).  The VS1 is built out using rack-mount, Proliant hardware for both the server hosts and for the storage and features a P4000 series iSCSI storage array.  It is rated to handle up to 750 virtual machines and can scale up to 8 physical hosts.  The VS2 is built out using HP BladeSystem with a P4800 iSCSI storage array (covered in depth last week).  It is rated for up to 2500 virtual machines and can scale up to 24 physical hosts.  The third offering is the VS3 which is built on HP BladeSystem and the 3PAR Utility Storage platform to provide ultimate scale and performance.  VS3 introduces fiber channel storage capability and scales up to 6000 virtual machines with up to 64 hosts.

In terms of choice, VirtualSystem supports all three major hypervisors from VMware, Microsoft and Citrix.  Using my company as an example again, the multi-hypervisor datacenter already exists.  We are utilizing VMware vSphere heavily and then some Citrix XenServer.  When it came to planning upgrades for our aging MetaFrame/XenApp farm, we looked at virtualization.  As we evaluated XenServer, we found it to be “good enough” for running Citrix XenApp on top of it.  XenApp has its own failover and redundancy built into the application layer, so many of the VMware advanced features did not matter.

For VirtualSystem, HP is also handling all support for both the hardware and software for these solutions.  Having experience with HP’s software support teams, I can report that they do a good job at it.  I would not say they are always perfect, but in general, they have solved our issues and advised us well, so in reality this is a big benefit.  For those who want not on break/fix support, HP offers Proactive 24 Services for an additional level of preventative support.

Building to the cloud – HP CloudSystem

As I learned at HP Discover, just because you have a large virtualization pool in your datacenter does not mean you have a private “cloud.”  The critical difference between a virtual infrastructure and a cloud is orchestration and automation.  Built on top of HP VirtualSystem, HP CloudSystem is a solution that offers all of the necessary orchestration, service catalog and workflows to turn virtual infrastructure into a true cloud.  There is a clear and clean upgrade path from VirtualSystem into CloudSystem.  And for those starting fresh or who want to evaluate the HP solution, there is even an HP CloudStart service which will deliver a rack with CloudSystem into their datacenter and have it fully operational in 30 days or less.

CloudSystem is offered in three levels: CloudSystem Matrix, CloudSystem Enterprise and CloudSystem Service Provider.  CloudSystem Matrix is targeted towards those looking to automate the private cloud, customers who are looking to add automation and orchestration to their existing virtual systems.  It provides infrastructure as a service (IaaS) and basic application provisioning in minutes.  CloudSystem Enterprise extends upon Matrix and allows for private and hybrid cloud, enabling the bursting of workloads to public cloud.  It is a platform for hosting not only IaaS, but Platform as a Service (Paas) and Software as a Service (SaaS).  CloudSystem Enterprise provides application and infrastructure lifecycle management and allows for management of traditional IT resources in addition to virtualized resources.   The CloudSystem Service Provider edition extends upon the Enterprise edition and allows for multiple tenants on a single infrastructure, securely without exposing customer data between customers.  It is intended to host public and hosted private clouds for customers.  The editions in CloudSystem are more about capabilities and less about limits, compared to VirtualSystem.

Since automation and orchestration is the key of CloudSystem, that is where I wanted to focus.  The base of CloudSystem is the Matrix Operating System, which is the same combination of HP software found in the HP VirtualSystem solution.  On top of the Matrix Operating System, the CloudSystem Matrix solution includes Cloud Service Automation for Matrix.  This software includes Server Automation for lifecycle management for physical and virtual assets via a single portal and set of processes and HP SiteScope, an agent-less monitoring solution for performance and availability.

The enterprise and service provider editions include a beefed up version called, simply, Cloud Service Automation.  It includes the entire orchestration, database and middleware automation pieces of the pie and a cloud controller software.  These additional pieces allow not only the automatic and streamlined provisioning of physical and virtual servers but also the provisioning of the required glue that sits in between the apps and the servers.  The diagram below from HP shows all the moving parts of Cloud Service Automation better than I can explain in words.  And because, Cloud Service Automation is total lifecycle management, there are the pieces for monitoring and performance management which would be needed.  In addition, the centralized portals serve as point for both end users and IT professionals to manage the cloud.

Cloud Maps are another feature of CloudSystem and these are predefined automation workflows for deploying software and platforms easily.  These are the piece of the puzzle that allows for improved deployment times and also allow for drag and drop creation of new workflows and processes in the cloud.  HP has worked with its software partners to create these maps of requirements and automate the process of deploying their solutions.

Beyond all of the capabilities, HP is working hard to make this an open solution by making it compatible to burst workloads into third party clouds, whether its Amazon’s EC3 or a vCloud service provider.  This was a point stressed during the announcements at HP Discover and during the call on Tuesday.

This is post number two for Thomas Jones’ Blogger Reality Show sponsored by HP and Ivy Worldwide. I ask that readers be as engaged and responsive as possible during this contest.  I would like to see comments and conversations that these entries spark, tweets and retweets if it interests you and I also request that you vote for this entry using the thumbs up/thumbs at the top of this page.  As I said earlier, our readers play a large part in scoring, so participate in my blog and all the others!

This isn’t the first time I’ve written about CloudSystem.  In June,  I posted about my take on CloudSystem Service Provider from a potential service provider’s perspective.  I encourage you to take a look at that post, too, after you take a minute to comment and/or vote on this post.

Overview and experiences with HP BladeSystem.

For anyone who found this post looking for my week two post for the Blogger Reality Show, please see the Simplifying IT support and deployments with converged systems post instead.  This post is just more or less general info about HP BladeSystem and my experiences.  

I worked with HP to deploy two HP BladeSystem C7000 chassis in my work datacenters about three years ago.  Our group has two enclosures in to separate datacenters which are mirrors of each other in configuration.  At first, we only ran Windows clusters on the BladeSystems with one node in each enclosure.  We considered this a safe way to get familiar with the technology, management and understand the reliability of the BladeSystem in our environment.  We, as others have, did find the enclosures to be very reliable, eventually reconfiguring them to add Virtual Connect Flex-10 and moving some of our ESX onto BladeSystem.  Today, HP BladeSystem runs a very large percentage of our overall infrastructure.

Some of the BladeSystem advantages we found are:

  • Reduced energy costs and better energy management by collapsing multiple servers into a single enclosure
  • Reduced footprint in the datacenter and increased density
  • Reduced cabling and no need to re-cable enclosures if using the VirtualConnect technology (wire once)
  • Condensed space required in the datacenter – in a C7000 chassis, up to 16 servers in 10U of space.  In the past, 16 servers would require a minimum of 16U (1U servers) and worst case of up to 64U (4U servers) in the rack
  • Allows users to mix and match workloads and types of servers while keeping management tools the same

Just as with the Converged Storage solution I covered last week, it is a small incremental cost to scale up, after the initial chassis investment.  The incremental cost to add a server is much smaller than buying an entire rack mount server.  And we reap the benefits of buying the newest, fastest servers with the most memory possible as those limits increase and we only buy the blades as we need them.

It appears that management was a primary focus when HP created the BladeSystems.  The OA or Onboard Administrator is the primary resource for configuring, managing and monitoring the chassis.  From the OA, you can manage the system IP addresses, configuration of the blades and interconnect bays and launch into other management interfaces.  The standard iLO or Integrated Lights Out management processor, which is a separate processor found on all Proliant servers (rackmount or blade) is the individual server blade administration point and it is very well integrated into the OA.  For those customers who choose HP’s Virtual Connect technology, you can launch into the Virtual Connect Administration panel from within the OA, too.  The VC administrator is where you configure virtualized MAC address and Fiber Channel WWID assignments which allows you to create a network and SAN profile for a particular server which contains virtual addresses assigned to the blade.  If the blade has a hardware problem, the profile can be reassigned and the blade can be booted to another blade, if one is available.  In addition, Insight Control is available which allows for enhanced automation of deployments and management.

The only downside I can report is that we have found firmware upgrades to be fairly invasive and disruptive due to the amount of infrastructure that has been condensed into a single enclosure.  Because of this, we have designed the workloads on the BladeSystem so that we can fail-over workloads from on enclosure to the other with minimal disruption.  We use Microsoft Failover Clusters, vSphere (which allows vMotion of workloads), and XenApp which allows us to put hosts in maintenance mode and move users off of them.  The firmware process has improved since HP introduced the concept of release sets for BladeSystem firmware last year, but it still takes some coordination on our part.

HP BladeSystem is the market leader in blades according to IDC.  It has consistently held the number one spot in blade market share quarter after quarter, with 50% of the blade servers shipped during Q1 2011.  It is not just market fluff or creative interpretation when HP says they are the leader in blades.

The HP BladeSystem has been a rock solid solution for our needs.  I commented last week to my boss that by far our most capable servers in the datacenter are in our blade centers.  Where blade computing began and is today is a huge leap.  There is almost nothing that cannot fit into a blade server.

Insight Control for vCenter (Recap HP Discover #4718)

HP Discover 2011 Give Away

A couple years ago at VMworld, HP announced a new plugin for vCenter, called Insight Control for vCenter, which pulled from HP System Insight Manager and on-board intelligence of ESX host Proliant servers and displayed system status within vCenter.  Since the introduction, the Insight Control for vCenter package has grown to include a second plugin for storage, introduced in May of 2010.

The two plugins are separate and users have the choice of installing one or the other on the vCenter server when running the installation.  As far as licensing is concerned, the good news is that the storage plugin is completely free of charge, but the server plugin is licensed as part of the Insight Control package, though there is no license check.  The server plugin is a separate package not included with the other Insight Control installation.  To obtain the installation media, go to

The goal of Insight Control for vCenter is to enable vCenter administrators a single pane of glass, which administrators are already using to manage vCenter.  By integrating the server and storage components, HP is delivering a package which also adds its own alerts to the normal vCenter alarms framework.  The plugin is very vCenter centric and uses the same user and permissions, adding its own permission groups which can be given to users.

The latest version of Insight Control for vCenter includes the ability to look into Virtual Connect Ethernet on BladeSystem and allows you to visually see how VM’s map back within the Virtual Connect area and to the external ports on the interconnect bay.  This is a big improvement for customers running Virtual Connect and ESX together, which has been a bit of smoke and magic which we couldn’t peer into very much.

The plugin also provides administrators with hooks into the other management software for HP.  Within the server module, there are links for additional information into HP System Insight Manager (SIM), to the iLO card and to the OA from different locations to gain full visibility.  The plugin does not try and replicate all functionality of these other management interfaces.  The same is true for the storage plugin, which includes hooks into CommandView EVA (and I assume other management interfaces, although I have to say I have only seen it demoed on EVA).

The storage plugin includes the ability to provision storage (if allowed by security permission within the plugin and vCenter) directly within vCenter.  It allows the ability to create clones and offload the work of creating the clone to the array instead of ESX.  In addition, the storage plugin also reports paths and references LUNs back to their VMFS datastores and correlates data which was previously difficult to correlate between the storage management and vCenter.

Insight Control Storage for vCenter currently supports all of the current generation of P2000 MSA, P4000 Lefthand, P6000 EVA, XP and P9500 series storage arrays.  Many previous generation EVA products are also supported.  HP is working to integrate support for 3PAR arrays into the storage plugin.

To setup and configure the Insight Control for vCenter, the plugin adds a new icon to the home screen in vCenter where all the passwords and integration can be setup.  Configuration for the storage plugin is configured from within the storage tabs on the ESX hosts.  Once configured against the array or array management software (CommandView EVA, in our demo and our shop), the list of disk groups is presented and the administrator can allow or disallow write access and provisioning to the arrays from within the vCenter plugin.  Save that configuration and the plugin is ready to begin querying the management servers.  Once it populates a view of the storage environment, the tabs populate with a good amount of data.  Data is a cached local copy of data retrieved from the storage management, so a refresh button exists in the storage plugin to pull updated information if changes are made outside of the Insight Control for vCenter plugin.

Of note with the plugin, the system information pulls from both the iLO card on the servers, from HP System Insight Manager (SIM), and with the Onboard Administrators of BladeSystems.  Without HP SIM, some information is not populated.  If you have a host without the Insight Agents loaded, you will only get a partial view of the server and some details will not populate.  For ESXi hosts, the Insight Control plugins use CIM (Common Information Model) to pull the system information.  HP also releases a custom release of ESXi which includes all the Insight functionality baked into the ESXi distribution.

In the interest of full disclosure, HP and Ivy Worldwide invited me and paid for my trip to HP Discover.  Even though, I am trying to relay the information as impartially as possible.