This is the last note I took from LISA 2010 conference. It’s a great talk by Loren Jan Wilson drawing his experience with vendors while working at a super computer center.
The super computer, Intrepid, consists of 40,960 nodes on 40 racks. Each node has 4 core CPU. Of all the nodes, 640 nodes are dedicated for I/O. There is no local storage at each node. The super computer links to a very large tape library for archiving.
While operating the super computer, the speaker had some issues with high-speed network switches, e.g. 6% random port death, 15% quad port flaky but never fail 100%. To complicate the issue, there is no log and CLI for troubleshooting, but Web interface.
I believe the trouble the speaker faced before is not a single case in the industry, and never be. As long as you have to buy equipment/software from vendors, there will be issues one way or the other. A great thing the speaker did is to summarize and share the tips on how a customer should work with an IT vendor for a successful IT project.
I find these tips are very helpful, and think customers and vendors should all know about them as listed below: