Page 2 of 2

Re: NAS goodness!

Posted: Fri Jul 06, 2012 11:31 pm
by Dan
Ned - I will check and get back to you regards freight to NZ. I have no idea if the arrays can take SATA drives as all the ones I have are FC (i think).

insane - comes with two SP's, not sure what is meant by the term FC though and as for licenses, I have no idea.

Re: NAS goodness!

Posted: Sat Jul 07, 2012 1:12 pm
by insane
Oh my comment about whether there were any fiber channel switches was just to try understand how these were hooked up. Typically these would be setup with a pair of brocade switches or similar to form a SAN so you can connect multiple servers to the storage, but if not then that's OK too as it's still usable as these can do iSCSI too.

If the shipping to NZ isn't too crazy I'd be keen for sure :)

Re: NAS goodness!

Posted: Sat Jul 07, 2012 8:30 pm
by johu
Ned wrote:So if i understand this correctly, these take SATA drives as well as FC drives?
You can't mix and match FC/IDE/SATA/EFD disks on same shelf. CX300 is old model dating back to 2004-2005. If I remember correctly with new enough FLARE it supports 18/36/73/146GB FC, 320GB IDE (PATA) and 250/500GB/1TB SATA disks. There's plenty of small disks floating around but 500GB and 1TB ones are very expensive. CX300 era devices can't use same capacity SATA disks intended for newer models. Entire DAE with disks can be reused on newer arrays, but it's not usually worth it given capacity, slow 2Gbit/s FC backend slowing down all DAEs on same FC backend loop and expensive service contracts.

There's different type of DAE for FC, IDE and SATA disks on CX300. You can't replace disks except with exact EMC supplied models listed on EMCs compatibility list, they have special EMC firmware and 520 byte sectors. 512 bytes for data and 8 bytes for ECC to reconstruct corrupted sectors without involving RAID. That happens and is considered normal, with 60+ disks I've seen around 5 ECC repairs triggered per year. Systems that don't implement ECC, such as Linux servers, those same errors happen but go unnoticed and data gets corrupted. That's one of reasons why ZFS is so great compared to other filesystems/volumemanagers. Trying to re-use disks outside EMC array might be tricky as well due sector size.

CX300 PATA and SATA disks have small interposer circuitboard on back of them. Connector looks like FC connector but it's something different. I've disassembled one SATA DAE2 and it seemed that LCC's have four pcs of four channel PATA controllers. Then there's SATA-PATA bridges and SATA signalling thru FC connector backend to interposers on disk trays and finally to actual SATA disk. My guess is that with PATA disks there's second SATA-PATA bridge on interposer. LCC itself got CPU (under heatsink, dunno model), some RAM and flash plus FC chip. It's then bridging this PATA->SATA->PATA chain to FC. Awful solution and also reason why disks aren't interchangable with newer DAEs.

Newer EMC CX series models use different DPE that can use FC/SATA/EFD (EFD=Enterprise Flash Device, SSD) but still only FC/EFD mixing, no FC/SATA or EFD/SATA.

Also remember that you'll need FC switches and FC cards on your servers to connect this storage. Ethernet ports are only for management ie. no NFS, no iSCSI, no SMB etc.

When things work it's very trouble free system. When things go wrong... well you better have good backups or plenty of time and skills. These can be very frustrating devices to work on if you don't have good knowledge and preferrably good relations to someone on EMC support that can provide you help. Basic procedures are documented on EMC web and firmware updates are available to customers with active support contract. Tools to recover from bigger issues and related documentation is only available to authorized EMC service partners (Unisys etc) and not to end users / customers - even if they have valid contracts that this device likely doesn't. I've seen Unisys service technician bootstrapping Windows (as these are running Windows) over serial port to CX4 that got hosed during FLARE upgrade. Took over 12 hours to get it running with some big name 10+ year experience EMC tech guys on phone / remote connection. Good thing that there was no downtime. Dual everything setup so VMware hosts were served by working SP until failed one was recovered.

Hardly known feature what one can do with these old CX DAEs is to ditch DPE ("brain" DAE with SPs) and chain DAEs with disks directly to server using FC. There's some company on ebay selling optics that turn copper FC on back of DAE to optical FC with LC connector. This can then be connected to server using fiber. Server will see each disk on DAEs as separate disk and one can then make software raid using ZFS on them and export it over FC or Ethernet to other servers. Even multipathing works with that hack. Essentially once expensive CX300 becomes cheap JBOD DAS. Too bad even that's completely useless these days when single 4TB USB disk can store as much as entire 42U rack of 36GB or 73GB FC disks did back in 2004.

EDIT: Stratos Lightwave MHSDC-8-8-1-L (2Gbit HSSDS copper FC to 2Gbit optical FC) is adapter one needs to turn EMC DAE to JBOD DAS. Rare part, goes for about USD 40 on ebay time to time. No idea for what purpose these were originally manufactured back in 2004 (based on manufacturing date on ebay ad).

http://www.cs-electronics.com/images-large/mia-5000.jpg
http://www.ebay.com/itm/ws/eBayISAPI.dl ... 1102367586

Re: NAS goodness!

Posted: Sun Jul 08, 2012 12:14 am
by Fred
Wow! That was a great read! Thanks for all of the insight! I guess if there is a lot of proprietary hardware floating around that would need drivers a Linux retrofit might be out of the question. I'm still a bit acronymed after all of that, but I think I'll recover :-)

Re: NAS goodness!

Posted: Sun Jul 08, 2012 9:44 am
by johu
Fred wrote:Wow! That was a great read! Thanks for all of the insight! I guess if there is a lot of proprietary hardware floating around that would need drivers a Linux retrofit might be out of the question. I'm still a bit acronymed after all of that, but I think I'll recover :-)
Running Linux on SP should be doable as x86 Windows already runs on them. Problem is amount of custom asics, busses between SPs used to synchronize caches, passing traffic received via "wrong" SP to one with active lun etc.

EMC Clariion FC and CX series have long history. It's actually Data General product which is why still even today SCSI probe returns "DGC" (Data General Corporation) as manufacturer on LUNs presented to servers. Also name "CLARiiON" was chosen because it rhymed with DGC product called "AViiON". Aviion in turn was designed to replace "Nova" and therefore project name was "novaII" -> change order and case of letters and you have "AViiON". :)

There is somewhere, or at least was, fascinating series of blog posts by ex-DG guy explaining Clariion history. Why it was originally created, how in-house OS evolved and how challenges with aging OS eventually lead them to switch to Windows NT for FC series (predecessor of CX). Ending with downfall of DG and finally EMC buying them to gain access to Clariion technology.

Re: NAS goodness!

Posted: Sun Jul 08, 2012 1:31 pm
by Fred
You should link it, if you know where it is! :-)
Fred wrote:a lot of proprietary hardware floating around that would need drivers
johu wrote:custom asics, busses between SPs used to synchronize caches, passing traffic received via "wrong" SP to one with active lun etc.
Exactly what I was thinking.

Fred.

Re: NAS goodness!

Posted: Sun Jul 08, 2012 6:17 pm
by johu
After digging my old email archives I think those blog posts were made by Steve Todd. This one contains DG era history. Plenty of other posts worth reading over there for those interested.

Reason I had these on my emails is because EMC guys themselves claimed it's based on Linux. :) I might as well paste relevant parts of that email here. Links are all dead now as that was 4+ years ago. CX4, which was last in CX series before being replaced with VNX was based on 64-bit Windows XP Embedded. Genuine Windows license stickers on SPs say Windows Storage Server, but that's just marketing.

EMCRemote release notes:
- EMCRemote v4.02 is required for the CX700, CX500 and the CX300 Arrays. These arrays have a base O/S kernel that are based on Window’s embedded XP.
- EMCRemote v4.02 is backwards compatible with the CX600, CX400, CX200 and FC4700, whose base O/S kernels are based on Window’s embedded NT.

Microsoft KB:
Note: EMC CLARiiON Arrays run Embedded Windows in a locked down state that is optimized as an embedded device for block level data storage functionality. EMC does not recommend or support joining the CLARiiON SPs to a Windows domain.

Blog comments:
Dale: Dear sir, is it true that the Clariion runs Windows XP embedded?
Steve Todd: Yes it is true that today's CLARiiON is running a version of Windows XP embedded. CLARiiON made the transition to a Windows-based implementation in the late 1990s.
Dale: Kind thanks, gent. So does this imply that FLARE runs as a Win32 application?
Steve Todd: FLARE currently runs as a driver in the kernel.

Steve Todd. EMC Distinguished Engineer Steve Todd is in his third decade of storage software development, and has generated over 140 patents and patents pending during that time. He writes about his experiences building software products for the storage and information industry.

Q: Does FLARE still use a Windows-based kernel?
A: FLARE uses an embedded Windows XP kernel for system call and scheduler services. All of the rest of the functionality, including all I/O ports, and intellectual property in FLARE is and has been developed by EMC.

Re: NAS goodness!

Posted: Tue Dec 02, 2014 7:30 am
by xeno150
johu wrote:

Code: Select all

# Done. Enjoy your new PIX-OS 8.0(4) image without checks for image validity.
This is so awesome.
I found your description and tried it myself - however, my pix does fail on load:

Code: Select all

tftp pix804.bin.hacked@x.x.x.x............................................................
Received 7538688 bytes
Cisco Security Appliance admin loader (3.0) #0: Thu Aug  7 19:15:24 MDT 2008
################################################################################################
256MB RAM

Total NICs found: 2

An internal error occurred.  Specifically, a programming assertion was
violated.  Copy the error message exactly as it appears, and get the
output of the show version command and the contents of the configuration
file.  Then call your technical support representative.

assertion "0" failed: file "cpu_hog.c", line 189
No thread name
    vector 0x00000020 (user assertion)
       edi 0x00108c42
       esi 0x0010898a
       ebp 0x0009ff58
       esp 0x0009ff24
       ebx 0x000000bd
       edx 0x00000000
       ecx 0x0079b9a8
       eax 0x016575c0
error code n/a
       eip 0x01301c35
        cs 0x00000008
    eflags 0x00000046
       CR2 0x00000000

An internal error occurred.  Specifically, a programming assertion was
violated.  Copy the error message exactly as it appears, and get the
output of the show version command and the contents of the configuration
file.  Then call your technical support representative.

assertion "_vf_mode_init" failed: file "vf_api.c", line 99


-----------------------------------------------
Traceback output aborted.
Flushing first exception frame:
    vector 0x00000020 (user assertion)
       edi 0x00108c42
       esi 0x0010898a
       ebp 0x0009ff58
       esp 0x0009ff24
       ebx 0x000000bd
       edx 0x00000000
       ecx 0x0079b9a8
       eax 0x016575c0
error code n/a
       eip 0x01301c35
        cs 0x00000008
    eflags 0x00000046
       CR2 0x00000000
Nested traceback attempted via interrupt, from:
    vector 0x00000020 (user assertion)
       edi 0x00108c42
       esi 0x0010898a
       ebp 0x0009ff58
       esp 0x0009ff24
       ebx 0x000000bd
       edx 0x00000000
       ecx 0x0079b9a8
       eax 0x016575c0
error code n/a
       eip 0x01301c35
        cs 0x00000008
    eflags 0x00000046
       CR2 0x00000000

An internal error occurred.  Specifically, a programming assertion was
violated.  Copy the error message exactly as it appears, and get the
output of the show version command and the contents of the configuration
file.  Then call your technical support representative.

assertion "_vf_mode_init" failed: file "vf_api.c", line 99



Rebooting....
Could you please email your version, if possible?
That would be awesome!

Thanks a lot!

Re: NAS goodness!

Posted: Tue Dec 02, 2014 9:45 pm
by johu
xeno150 wrote:This is so awesome.
I found your description and tried it myself - however, my pix does fail on load:
Sorry don't have it anymore, but if you follow instructions I posted you end up with identical file. Since error you get is same what happens when booting unmodified firmware you probably skipped part where ascii string "506E" is changed to "XXXX" and "515E" to "506E". At least do that "MB RAM" to "MB-RAM" so you'll see your modifications are actually applied.


That EMC CX300 installed in early 2005 I mentioned on first page is still running strong - heard that yesterday. Not that it makes any sense given how power hungry it is when you get same capacity from single USB drive these days. It's turning 10 years within few months. :)

Found specs and price on ancient email.

Clariion CX300 base with "dual everything"
15 x 73GB 15k FC
15 x 250GB 7.2k SATA
2 x ds-24m2 FC switches
powerpath licenses
36mo 24x7 on-site with 4h response time
on-site installation
total 30650 EUR VAT 0%

Re: NAS goodness!

Posted: Wed Dec 03, 2014 7:10 am
by xeno150
johu wrote: Sorry don't have it anymore, but if you follow instructions I posted you end up with identical file. Since error you get is same what happens when booting unmodified firmware you probably skipped part where ascii string "506E" is changed to "XXXX" and "515E" to "506E". At least do that "MB RAM" to "MB-RAM" so you'll see your modifications are actually applied.
You're absolutly right. I completly forgot that. I will try again later.
Thanks a lot! :)

And for the sake of documentation, lzma has been replaced by another - more recent lzma version.
The "old" version which has to be used is lzma_alone
( sudo apt-get install xzdec lzma_alone )
Which results in the commands:

lzma_alone d pix804.bin.lzma pix804.bin.uncompressed

lzma_alone e -a1 -d24 pix804.bin.uncompressed-hacked pix804.bin.lzma-hacked

johu wrote: That EMC CX300 installed in early 2005 I mentioned on first page is still running strong - heard that yesterday. Not that it makes any sense given how power hungry it is when you get same capacity from single USB drive these days. It's turning 10 years within few months. :)
Wow! Now that's cool!
Yeah, it doesn't make any sense - but I love reworking old machines and using them: There is so much to learn from old stuff, as in those days, there were real innovations and not just buzzwords ;)!