Category Archives: SAN Management

EMC World 2011 – Las Vegas – day 1

So after the first day at EMC World what Marvels of technology have been announced ?
What groundbreaking nuggets of geeky goodness to be announced. So, first things first VPLEX ! looks like they may have cracked it..   Active/active storage over a synchronous distances, Geoclusters will never be the same again !!..   and also a slightly ambiguous announcement around integration with Hadoop opensource (more to follow on that).

What was the message of the day though ? What was this years theme..   This year EMC are talking about Big data and the cloud. Clearly recent acquisitions of Isilon and Greenplum have planted EMC’s head firmly back in the clouds.  Greenplum giving end users the ability to scale out Database architectures for data analytics to mammoth scale with Greenplums distributed node architecture and massive parallel processing capabilities. To br frank, learning about the technology was borderline mind numbing, but my god its a cool technology. Then we have large scale out NAS with Isilon and its OneFS system giving the ability to present massive NAS repositories and scale NAS on a large scale. So obviously, EMC are talking about big data.

I also had the opportunity to sit in on an NDA VNX/VNXe session and what they’re going to do is….    aaah, I’m not that stupid. But needless to say, there are some nice additions on the way, the usual thing with higher capacity smaller footprint drives and getting more IO in less U space, but also some very cool stuff on the way which will enable EMC to offer a much cheaper entry point for compliance ready storage..  watch this space.

In true style EMC threw out some interesting IDC touted metrics further justifying the need to drive storage efficiencies and re-iterating the fact that there will always be a market for storage. So, our digital universe consists of 1.2 Zettabytes of data, currently, of which 90% of that is unstructured data and that figure is predicted to grow by x44 over this decade. Also 88% of fortune 500 companies have to deal with Botnet attacks on a regular basis and have to contend with 60 Million Malware variants.  So making this relevant, the 3 main pain points of end users are; firstly our time old friend budget, then explosive data growth and securing data.

So how have EMC addressed these ? Well, budget is always a fun one to deal with, but with efficiencies in storage by way of deduplication, compression, thin provisioning and auto tiering of data, end users should get more bang for their buck. Also, EMC easing up on the rains with pricing around Avamar and the low entry point of VNXe, this should help the case. Dealing with explosive data growth again tackles with deduplication, compression, thin provisioning and auto tiering of data, but also now with more varied ways of dealing with large sums of data with technologies such as Atmos, greenplum, Isilon. Then the obvious aquisition of RSA to tie in with the security message, all be it that has had its challenges.

I’m also recently introduced the concept of a cloud architect certification track and the concept of a Data Scientist (god knows, but I’ll find out). So I went over to the proven professionals lounge and had a chat with the guys that developed the course. Essentially it gives a good foundation for steps to consider when architecting a companies private cloud, around Storage, virtualisation, networking and compute. If you’re expecting a consolidated course which covers the storage consolidate courseware, Cisco DCNI2, DCUCD course and VMware install configure manage,  then think again, but it does set a good scene as an overlay to understanding these technologies. It also delves into some concepts around cloud service change management and control considerations and the concept of a cloud maturity model (essentially EMM, but more cloud specific). I had a crack at the practice exam and passed with 68%, aside from not knowing the specific cloud maturity terms and EMC specific cloud management jargon anyone with knowledge of servers, Cisco Nexus and networking, plus virtualization shouldn’t have to many issues, but you may want to skim over the video training package.

There was also a nice shiny demo from the Virtual Geek Chad Sakkac showing the new Ionix UIM 2.1 with Vcloud integration using CSC’s cloud service to demonstrate not only the various subsets of multi tenancy, but also mobility between disparate systems. When they integrate with public cloud providers such as Amazon EC2 and Azure, then things will really hot up, but maybe we need some level of cloud standards in place ?…   but we all know the problem with standards, innovation gives way to bureaucracy and slows up…   but then again with recent cloud provider issues, maybe it couldn’t hurt to enforce a bit of policy which allows the market to slow up a little and take a more considered approach to the public cloud scenario..   who knows ?

Anyway.. watch this space..  more to come


Adminstration of Clariion with VMWare… Getting easier

So, EMC released the NFS plugin for VMWare to support storage administration tasks on Celerra from the VI Client a while back, which was very cool and had some very impressive features..    but what about the Traditional SAN man ?! 

Well, yesterday EMC announced a VMWare plugin for Clariion.. 

Product Overview

The EMC CLARiiON Plug-in for VMware simplifies storage administration between the VMware Virtual Center Server and CLARiiON storage systems. It offers end-to-end management of storage related tasks including provisioning of datastores, provisioning of raw device mapping (RDM) devices, and array-based virtual machine replication.

New Feature Summary 

The EMC CLARiiON Plug-in for VMware allows you to perform the following specific tasks directly from the VMware vSphere client:

  • You can provision new datastores (VMFS volumes) or raw device mapping (RDM) volumes
  • Delete existing datastores backed by CLARiiON CX4 storage
  • Creation of virtual machine replicas using array-based replication services
  • The plug-in also gives you the option to publish the replicated virtual machines to a View Manager.


·       EMC CLARiiON Plug-in for VMware is customer-installable.

·       EMC CLARiiON Plug-in for VMware requires CX4 storage systems running Release 29 FLARE.

 Thats all I have at the minute, but will be picking the brain of the EMC bods as I go to get some more info.

Very usefull feature though !!

Is your EMC Solution Supported ? Why not check ?

EMC are pretty good at making sure they test, test and test again when it comes to interoperability with other vendors. The EMC Elab enables you to make use of all that testing and check that your Storage environment is supported with EMC.

See below for a walkthough guide of the Elab storage wizard.

The Long awaited….

A few new things comming out on EMC’s mid tier storage range to look out for.  One of which is very discreetly named Project Odin and will make the life of EMC Celerra users and resellers alike a touch easier. Its a management console to manage Celerra and its respective back end Clariion rather than having to jump into navisphere to manage the Clariion directly ! from what I gather its will run on any DART or FLARE OS and is pointed at the system to which it needs to manage via IP address, then the appropriate profile is loaded to reflect the functions relevant to said Clariion/Celerra..  about time !!   as I understand it, there will be an announcement in feb..  but it won’t be going GA for a little while. Watch this space !!

EMC Clariion FAST.. Fly like a butterfly, sting like a bee… really ?

So the FAST suite is available on Clariion. Good news !! If you don’t know what FAST (Fully Automated Storage Tiering) is, in a nutshell it’s automated storage tiering (as implied in the name), it ensures that LUNs which may have critical performance requirements and variable characteristics in terms of IO utilisation are using disk as efficiently as possible. LUNs of data which are accessed frequently will be serviced by one tier of disk (ie solid state) and others by another (ie fiber channel disk or SATA) and all this is done dynamically on the fly using the Clariion Virtual LUN technology (meaning you can migrate a LUN from 1 set of disks to another seamlessly to the application and retaining all the properties of the LUN); All this is done on the automatically using FAST.  This is especially relevant now as virtualisation is rife and the ability to be so flexible by way of server deployment requires that the storage is either meticulously designed and frequently reviewed, adjusted, etc or the storage platform is adaptive and flexible (even more relevant in multi-tenancy environments which offer a managed service) . FAST enables EMC storage to fall under the latter.

So, is there a sting in the tail ? is it silly money ? much like Control Center was (although that did get better to be fair)..    surprisingly…   no. The FAST suite of tools is suprisingly well priced at a touch over £6000 list. Bearing in mind that the FAST suite isn’t just the FAST software, it also includes Navisphere Analyser (for analyzing and monitoring SAN bandwidth and IO),  QOS Manager (for performance tuning) and they throw in an assessment service to provide recommendations as to which LUNs are most suitable for migration to fast managed LUNs which will utilise EFD (solid state drives to the rest of us) and/or SATA etc..   (I’ll come back to why this is required a but later)   considering that you’re looking at a £10k list price for Navisphere Analyser and QOS Manager alone, thats not a bad deal. But then it wouldn’t be as you’re still looking at just under £8000 for an enterprise flash drive and FAST is as good a mechanism as any to drive sales of solid state drives.  But this isn’t just a smoke and mirrors mechanism to sell solid state drives, the benefits are real. The capital expenditure involved in deploying Enterprise flash drives with FAST may be undesirable to a lot of businesses, but the return on investment is again very real. The requirement to procure mass amounts of FC drives to support highly transactional databases is not gone, but certainly minimized, man hours required for certain laborious storage admin tasks is reduced (especially in environments may have applications with extremely variable disk loads), power and cooling requirements are reduced, the list goes on..

So why is there an assessment service ?  can’t I just chuck everything on FAST managed LUNs and tell it to go do ?…   Yes,  you could. But Solid state drives are still expensive, so make the best use of them you can.  So I might suggest that LUNs with a lesser performance requirement and predictable disk load characteristics sit on standard LUNs.

See below to see FAST on EMC’s VMax..   now just waiting for this on Clariion and Celerra (sub LUN level)

and before some boxing boff corrects me on my Muhammad Ali quotation in the post title. I know its “flout like a butterfly, sting like a bee”..  but cut me some slack, float didn’t quite fit.. call it creative license  😉

Virtual LUN migration rules with Clones

Something which came up recently was a question around using virtual LUN migration when Snaps or Clones are involved.

 When migrating a LUN using Clariions virtual LUN migration the following applys :

If migrating a LUN to a destination LUN of the same size, Source LUNs for Snaps are Clones can be migrated without issue.

However if migrating to a larger destination LUN, any Snapview sessions must be destroyed, Clone groups must be destroyed and re-configured after the migration.

Also, its worth noting, that snapview reserve LUNs or Clone private LUNs cannot be migrated.

More info on the attached EMC document.


Sizing for performance on Clariion

A few things I would suggest you do before just sizing a bunch of disks for capacity.

Firstly, your application response times are dependant upon a few things, one of the key things is ensuring you provision enough spindles/drives to support the disk load you are going to put on the SAN. If performance isn’t considered and the SAN isn’t sized with performance in mind, you could potentially see queue depth increasing on drives, the queue depth directly relates to the number of read/write requests waiting to access the drives. If the queue depth gets too high, applications which require sub 5ms or less response time (which a few do) may start timing out and you have problems. So you need to do a bit of data gathering..

In windows terms run something like perfmon in logging mode, looking at counters like bytes written, bytes read, number of reads, number of writes, queue depth. In Linux/unix terms, something like IOstat should be fine.

Ensure start logging over a reasonable period of time and ensure you capture metrics over peak hours of activity. We’ll come back to this in a minute.

Identify the profile of your data, which applications write to disk in a sequential fashion which write in a Random fashion. Sequetial writes are optimised by some clever bits the clariion does in cache, if you mix Sequential and random type data on the same RAID groups, you won’t see optimised writes with sequential data.

sequential data = Large writes (typically Backup to disk apps, archive applications, media streaming, large file)

Random data = lots of small read/writes. Ie. Database (exchange, SQL, oracle)

Next you need to think about the level of RAID protection required:

RAID 5 = Distributed parity, has a reasonably high write penalty, good usable vs raw capacity rating (equivalent of one drives usable capacity for parity) , a fair few people use this to get most bang for their buck. bear in mind that RAID 5 can suffer single drive failure (which will incurr performance degradation), but will not protect from double disk failure. EMC Clariion does employ the use of hotspares, which can be proactively built when the Clariion detects a failing drive and used to substitute the failing drive when built, although if no hotspare exists or if a second drive fails during a drive rebuild or hotspare being build, you will lose your data. write penalty = 4

RAID 3 = Dedicated Parity disk, great for large files/media streaming, etc. Although RAID 5 can work in the same fashion as RAID 3 in the right conditions.             

RAID 1/0 = Mirrored/Striped, lesser write penalty, more costly per GB as you lose 50% usable capacity to mirroring. RAID 1/0 provides better fault resilience and “rebuild” performance than RAID-5. It has better overall performance by combining the speed of RAID-0 with the redundancy of RAID-1 without requiring parity calculations. write penalty = 2

RAID 6 = Again distributed parity but instead of calculating horizontal parity only (as RAID 5 does) also calculated Diagonal parity essentially protecting you from double disk failure, there is a greater capacity overhead than RAID 5, but not as great as RAID 1/0 (equivalent of 2 drives usable capacity for parity).  The write penalty for RAID 6 is greater than RAID 5, although typically RAID 6 should only be used for sequential type data (back to disk, media streaming, large file), so writting to disk will be optimised by write coallescing in cache, writing all data and parity out do disk in one go without calculating parity on the disk (this incurring a lesser write penalty). This will only happen if the write sizes are greater than the stripe size of the RAID group and the drives are properly aligned.

There are a number of documents on powerlink outlining RAID sizing considerations for specific applications. Rule of thumb us keep your log files on seperate spindles to your main DB volumes.

I’m going to cut this a bit short –

from the data you gathered (mentioned at the beginning of the document), for each logical drive currently local or direct attached take the following :

number of reads + (number of writes x write pentalty) = disk load  (write penalty is specific to the raid type being used, RAID 5 = 4, RAID 1/0 = 2 )

Each drive type has an IOP rating = 10k FC = 150 IOPS, 15k FC = 180 IOPS, SATA = approx 80 IOPS

divide the disk load by the IOP rating of the disk type you’ve chosen and that will give you the spindles required to support your disk load (excluding parity drives). You will most likely have multiple volumes (LUNS) on a given RAID group, so ensure you divide the aggregate disk load of the volumes to reside on the given RAID group by the IOP rating of the drives in question.

Thats a starting point for you and some food for thought..   for more detail, there are FLARE best practice guides and application specific guides galore on powerlink…

Expanding a RAID Group / LUN on EMC Clariion CX4

A document I put together on expanding a RAID Group/LUN on Clariion CX4: (Click link below)

LUN expansion

EMC Clariion CX4 Virtual Provisioning

The EMC Clariion CX4  is a very flexible box. One of the ways which it enables admins to get the most bang for their buck is by utilising Virtual (Thin) provisioning. This effectivelly enables an admin to create a pool of storage from which smaller volumes of storage (thin LUNs) are provisioned. See below: 


(Click Image to enlarge) 

Many people will see thin provisioning as the answer to all their problems in terms of storage management.. No. It still needs to be monitored, managed and some intelligent thinking still needs to go into what volumes will be thin provisioned. The EMC Clariion CX4 also has some limitations around thin provisioning.  

Each model of Clariion has a limit as to how many drives may be contained in a pool of storage for virtual provisioning, so If you’re thinking of positioning a CX4-120 with one big pool of disks (ie 30 disks) for thin provisioning, think again.. please see below : 


Also please bear in mind, that although a disk pool for thin provisioning may contain more drives than RAID allows, the maximum size of a single LUN may not exceed 14TB.