Thursday, April 10, 2008

Fast Solution to Get a High Paying Job in the Best Technology Company

Is there a fast solution to get a Good job?

Can I somehow get a very good salary job fast?

What should I do to get that dream job soon ?

These are some of the questions many people email me with & people search frequently online. I am trying to list solutions,answers for these questions.

1.Best Time Management : Get to know Which company pays highest & Which profile does it pay a very good salary for.

2.Learn Fast Retain most : If you already suit above profile then you are lucky if not then you need to learn lots of stuff fast & also should retain a lot. More than anything a company wanna hire an experienced expert & not some one very good in theory. So u need to work fast, work smart & then learn from others experience. Internet helps u a lot in this. An example is this blog which has lots of interview questions highlighting what topics to stress more.Also videos which explain & show the things. In future I will write about tech topics along with their practical configuration etc which are practical.

3.Have a good Support System : Who can get you inside information or can help you get some advantage like who can teach you Server administration in a crash course, who can let you know which company is hiring in your nearby place, who can help you know how you can setup your own scaled down storage lab etc etc . You basically need someone who is more than a good friend.

4.Stick to the goal & be very positive : Results will not happen in a day. So you need to have consistency in your efforts. If I say read all 100+ posts on this blog & you will get a job for sure - how many of this blog's readers will try that. I am sure not even 1 person can have the patience to read all 100+ posts on this blog. But to my amazement a guy from Japan has gone through all the posts on this blog & taken print out made it as a small reference book. He emailed me yesterday saying that he got a nice job since he used this blog as his support system. Yes for him this blog served the purpose of being more than a friend. I am glad someone made a very positive use of the amount of knowledge available on this blog.

VMware Virtual Infrastructure 3 ESX Server Virtual Center Server Interview Questions

Vmware Technical Interview Questions. I was able to compile good list of 100 Vmware Interview Questions with Answers. Since Vmware is a hot technology & there are lots of job opportunities opening up for Vmware ESX administrator,Vmware Certified Professionals: lot of people are searching for "vmware interview questions" in Google.
I checked this keyword on Google & did'nt find any other website or blog giving info other than this blog "Storage Jobs Blog". Though there are lot of Vmware professionals they are too experienced & they feel this thing called "Interview Questions List" is not a big deal. They have such a cool years of experience & knowledgebase on Vmware that they are busy with their Virtualization project implementation & they DO not have time to attend any interviews. These guys in Vmware Virtualization domain are paid so well that they DO NOT have to worry about changing job or changing company.

Please find below 10 technical questions on Vmware ESX Server , Virtual Center & Virtual Infrastructure 3. More Questions with answers will be posted soon. But apart from this list if you want a single document of 100 Interview Questions with Answers - click on the Paypal Big button at the right side of this blog. This document containing 100 "Vmware technical Interview Questions & Answers" will never be published anywhere it is available for only people who are willing to pay $100. Yes the cost of this document is $100 . Along with this doc you will get several PDF files which detail many technical aspects of Vmware Virtualization which will be given FREE. Anyone who can clear & get a job in Vmware virtualization is going to get a pay of more than US $6000- $9000 Per month. So the price of $100 is less compared to the advantage U have by buying this. Email roger.smithson@gmail.com for more info.

Below are 10 Interview Questions with Answers on Installation and Upgrade of ESX 3.0.1 and Virtual Center 2.0.1

Installation and Upgrade of ESX 3.0.1 and Virtual Center 2.0.1

1.List the major components of Vmware Infrastructure ?
* The major components of VMware Infrastructure are:
ESX Server host .
Virtual Center Server.
Virtual Infrastructure (VI) Client .
Web browser.
License server.
Database.

2. What are the minimum H/W requirements to install VirtualCenter Server ?
* VirtualCenter Server hardware must meet the following requirements:
Processor : 2.0GHz or higher Intel or AMD x86 processor. Processor requirements can be larger if your database server is also run on the same hardware.

Memory : 2GB RAM minimum. RAM requirements can be larger if your database is run on the same hardware .
Disk storage :Nearly 1GB free disk space
Networking : 10/100 Ethernet adapter minimum (Gigabit recommended).
Scalability : A VirtualCenter Server configured with the hardware minimums can support 20 concurrent clients, 50 ESX Server hosts, and over 1000 virtual machines.

A dual processor VirtualCenter Server with 3GB RAM can scale to 50 concurrent client connections, 100 ESX Server hosts, and over 2000 virtual machines.

3. Which softwares are supported to install Virtual Center Server Software ?
* The VirtualCenter Server is supported as a service on the 32 bit versions of these operating systems :
The Virtual Center installer requires Internet Explorer 5.5 or higher in order to run.
o Windows 2000 Server SP4 with Update Rollup 1 (Update Rollup 1 can be downloaded from Windows XP Pro (at any SP level)
o Windows 2003 (all releases except 64 bit)

Virtual Center 2.0 installation is not supported on 64 bit operating systems.

4. Which Databases are supported to VirtualCenter ?
* Virtual Center supports the following database formats:
o Microsoft SQL Server 2000 (SP 4 only)
o Oracle 9iR2, 10gR1 (versions 10.1.0.3 and higher only), and 10gR2
o Microsoft MSDE (not supported for production environments)

5. What are the Hardware requirements for Virtual Infrastructure Client ?
* The Virtual Infrastructure Client hardware must meet the following requirements :
o Processor : 266MHz or higher Intel or AMD x86 processor (500MHz recommended).
o Memory . 256MB RAM minimum, 512MB recommended.
o Disk Storage . 150MB free disk space required for basic installation. You must have 55MB free on the destination drive for installation of the program .
o Networking . 10/100 Ethernet adapter (Gigabit recommended).

6. Which softwares are supported for Virtual Infrastructure Client ?
* The Virtual Infrastructure Client is designed for the 32 it versions of these operating systems:
o Windows 2000 Pro SP4
o Windows 2000 Server SP4
o Windows XP Pro (at any SP level)
o Windows 2003 (all releases except 64bit)

The Virtual Infrastructure Client requires the .NET framework 1.1 (included in installation if required).

7. What are the Requirements for VirtualCenter VI Web Access ?
* The VI Web Access client is designed for these browsers :
o Windows . Internet Explorer 6.0 or higher, Netscape Navigator 7.0, Mozilla 1.X, Firefox 1.0.7 and higher.
o Linux . Netscape Navigator 7.0 or later, Mozilla 1.x, Firefox 1.0.7 and higher.

8. What are the Minimum Hardware Requirements for ESX Server 3.x ?
* You need the following hardware and system resources to install and use ESX Server.
At least two processors:
o 1500 MHz Intel Xeon and later, or AMD Opteron (32bit mode) .
o 1500 MHz Intel Xeon and later, or AMD Opteron (32bit mode) for Virtual SMP.
o 1500 MHz Intel Viiv or AMD A64 x2 dualcore processors
1GB RAM minimum.
One or more Ethernet controllers. Supported controllers include:
* Broadcom NetXtreme 570x Gigabit controllers
* Intel PRO/100 adapters
For best performance and security, use separate Ethernet controllers for the service console and the virtual machines.
A SCSI adapter, Fibre Channel adapter, or internal RAID controller:
* Basic SCSI controllers are Adaptec Ultra160 and Ultra320, LSI Logic Fusion MPT, and most NCR/Symbios. SCSI controllers.
* RAID adapters supported are HP Smart Array, Dell Perc RAID (Adaptec RAID and LSI MegaRAID), and IBM (Adaptec) ServeRAID controllers.
* Fibre Channel adapters supported are Emulex and QLogic host bus adapters (HBAs).
A SCSI disk, Fibre Channel LUN, or RAID LUN with un partitioned space. In a minimum configuration, this disk or RAID is shared between the service console and the virtual machines.


9. Which Storage systems supports for installing and booting ESX Server ?

* ESX Server supports installing and booting from the following storage systems:
IDE/ATA disk drives Installing ESX Server on an IDE/ATA drive or IDE/ATA
RAID is supported. However, you should ensure that your specific drive controller is included in the supported hardware.
Storage of virtual machines is currently not supported on IDE/ATA drives or RAIDs. Virtual machines must be stored on VMFS partitions configured on a SCSI drive, a SCSI RAID, or a SAN.
SCSI disk drives . SCSI disk drives are supported for installing ESX Server. They can also store virtual machines on VMFS partitions.
Storage area networks (SANs) . SANs are supported for installing ESX Server. They can also store virtual machines on VMFS partitions.

10. Enhanced Performance Recommendations for ESX Server
* Some recommendations for enhanced performance:

RAM : Having sufficient RAM for all your virtual machines is important to
achieving good performance. ESX Server hosts require more RAM than typical
Servers : An ESX Server host must be equipped with sufficient RAM to run
concurrent virtual machines, plus run the service console.

Dedicated fast Ethernet adapters for virtual machines : Dedicated Gigabit Ethernet cards for virtual machines, such as Intel PRO/1000 adapters,iimprove throughput to virtual machines with high network traffic.

Disk location . For best performance, all data used by your virtual machines should be on physical disks allocated to virtual machines. These physical disks should be large enough to hold disk images to be used by all the virtual machines.

VMFS3 partitioning . For best performance, use VI Client or VI Web Access to set up your VMFS3 partitions rather than the ESX Server installer. Using VI Client or VI Web Access ensures that the starting sectors of partitions are 64K aligned, which improves storage performance.

Processors . Faster processors improve ESX Server performance. For certain workloads, larger caches improve ESX Server performance.

Hardware compatibility . To ensure the best possible I/O performance and workload management, VMware ESX Server provides its own drivers for supported devices. Be sure that the devices you plan to use in your server are supported.

Virtual Network Storage Architecture



Benefits of virtualization
- Significantly reduce downtime
- Improved performance: load spreading, balancing, multi-path, heuristic shifting
- Improve delivery and quality of storage services
- Simplify definition of storage policies and procedures.
- Managed file systems and volume mangers increase the scalability, reliability, security.
- Provides storage location and implementation transparency.
- Enables dynamic operations, storage systems reconfiguration and data location to change transparently to host environment.

Virtual Storage Appliance
- VTL (Virtual Tape Library) server.
- CDR (Continuous Data Replication) Backup/Recovery Server
- iSCSI server for Disaster Recovery
- Cluster System

Functionality of virtual storage subsystem
- Enhance the service quality of storage device/data, easier maintain system expansion, simplifies and centralizes the management of storage system, cut down the storage budget cost.
- Efficiently and simplify the functionality of following:
a. Data Backup and Restore
b. Cluster storage structure; enhance the capability of fault tolerance, I/O balancing, failover and fail-back.
c. 256 Snapshot Point in Time Data Replication
d. Data integration, migration and consolidation, simplify the management of storage system.
e. Disaster recovery, to rapidly resume system operation.
f. Load Spreading, Balancing, Caching, Security.

Above article courtesy:tekram.com/news/news_detail.asp?id=6

Fibre Channel Network Architecture brief introduction

FC Network protocol & architecture are other important topics any storage professional should be aware of.
Fibre Channel Protocol


Fibre Channel Overview
Fibre Channel is a networking standard that is designed to move data through specific devices at specific speeds. Initially, Fibre Channel was used primarily to attach servers to a storage device, such as a RAID array or a tape backup device. More recently, Fibre Channel has evolved as the architecture of choice for many storage area networks. Fibre Channel is ideally suited to this role because it is very reliable, very scalable, and very flexible. It's possible to add more storage without disrupting operations, use a number of backup schemes that don't load the main network, and manage very large data collections that span multiple disks.

Fibre Channel is a high speed (100 to 800 Mbps) medium used for data transfer and storage. Fibre Channel provides a logical bi-directional, point-to-point connection between a host and a device. It is essentially a serial data channel created over fiber optic cabling which makes it possible to transfer data at much greater distances than SCSI and ATA interfaces.

Fibre Channel is designed to work with either copper wires or with fiber. Although copper is used only for shorter distances, a variety of copper mediums can be used, including telephone wire or coaxial cable. If fiber optic cable is used, distances of about six miles can be achieved with minimal loss of speed. Another advantage of dedicated channel based communication is its ability to provide increased bandwidth than network based links.

Features and Benefits

Storage Area Networks are not new ideas in data storage, however they are increasing in popularity due to high demand by users who need to store large volumes of data. A SAN is a group of storage devices connected via a network of connections to host computer or server. The primary advantages of SANs are:
* Storage resource pooling/sharing
* LAN and server-free backup
* Centralized storage resource management
* Data sharing - Large number of users
* Performance - High Speed, Low Latency
* Distance (>10KM)

Architecture
Channels and networks are the two primary ways that data is transferred between devices. Fibre Channel uses channels and frames instead of packets to move data. Each frame is approximately 2KB in size. Of this amount, 1.5 percent is header information. Channels transfer data through switched or direct point-to-point connections and they work by creating a fixed connection between the source and destination devices until the transfer is complete.

Connections are made on Fibre Channel systems through "interconnection components" including switches, hubs, and bridges. The ability of Fibre Channel to use different interconnect devices makes it scalable depending on user needs. For small Fibre Channel networks, hubs and bridges may be used for connecting devices in a topology called Fiber Channel arbitrated Loop (FC- AL). As Fibre Channel networks get larger and network demands increase, switching may be implemented. A switched Fibre Channel network is called a fabric.

Fabric topology permits multiple paths between two ports on the Fabric. Loop topology, on the other hand, there can only be one active circuit at a time. Loop and Fabric topologies can be combined to provide both connectivity and performance. If a link in a point-to-point topology fails, communication between that pair of ports stops. Communication between other point-to-point connected Ports continues.

Fiber optics are excellent for transmitting data across a network because they are reliable. They do not have the same problems that are associated with copper cabling such as attenuation (loss of signal strength) and noise. It is also more secure than copper because crosstalk does not occur with Fiber Optic cables. Fibre Channel traffic can also be transmitted over other cable types such as copper, coaxial cables or Unshielded twisted pair (UTP) wires.

Class of Service - Fibre Channel technology makes use of classes of service to define communication between devices.

* Class 1 a dedicated channel between two connection devices. In this configuration, if a host and a device are connected, no other host can use that connection. The advantage of using service class 1 is speed and reliability.
* Class 2 is known as a "connectionless" service. It a frame-switched link that guarantees delivery of packets from device to device and packet receipt acknowledgments.
* Class 3 is called unacknowledged connectionless service and is good for broadcasts. This configuration allows multiple transmissions to be sent across the Fibre Channel fabric to multiple devices.
* Class 4 is called "intermix", which creates a dedicated connection but it also allows class 2 traffic to access the link. This method is very efficient and it allows for greater bandwidth because more than one connection can access the system at any time.



For more info u can read on ::www.lecroy.com/tm/Solutions/Protocol/fibrechannel.asp?menuid=

Storage architecture SAN architecture FC Network architecture

One of the Top question asked either in a Storage job profile interview or while attending a Client meeting is "Explain your system architecture". Based on whats their interest they may be interested to know about your
- Storage architecture
- SAN architecture
- FC Network architectiure

So this post has few images & some good links to help you learn more on these Very important topics

Storage architecture: Storage architecture design should follow a structured approach to ensure that the correct solution is adopted by the organization. The three basic types of storage architectures that are discussed in this blueprint(microsoft.com/technet/solutionaccelerators/wssra/raguide/ArchitectureBlueprints/rbabst_2.mspx) are:
- Distributed Storage
- Hybrid Storage
- Centralized Storage
Each of these storage architecture types defines a storage pattern that can be used as a starting point for providing guidance on how the storage should integrate with the business needs of an organization. If business needs are not communicated when enterprise architecture issues are being considered, it is easy for new projects to focus only on their own requirements and miss the wider picture that an architecture encompasses.

A structured design process for a complete enterprise storage solution consists of:
•Determining the storage requirements.
•Choosing the storage technologies.
•Defining fault tolerance technologies.
•Defining backup and recovery technologies.



Hybrid Storage Example


Centralized Storage Example


SAN Architecture example


Please read forthcoming posts for more photos & details about FC & Network architecture

Veritas Interview Questions with answers : Veritas Cluster Server

1. How do check the status of VERITAS Cluster Server aka VCS ?
Ans: hastatus –sum

2. Which is the main config file for VCS and where it is located?
Ans: main.cf is the main configuration file for VCS and it is located in /etc/VRTSvcs/conf/config.

3. Which command you will use to check the syntax of the main.cf ?
Ans: hacf -verify /etc/VRTSvcs/conf/config

4. How will you check the status of individual resources of VCS cluster?
Ans: hares –state

5. What is the service group in VCS ?
Ans: Service group is made up of resources and their links which you normally requires to maintain the HA of application.

6. What is the use of halink command ?
Ans: halink is used to link the dependencies of the resources

7. What is the difference between switchover and failover ?
Ans: Switchover is an manual task where as failover is automatic. You can switchover service group from online cluster node to offline cluster node in case of power outage, hardware failure, schedule shutdown and reboot. But the failover will failover the service group to the other node when VCS heartbeat link down, damaged, broken because of some disaster or system hung.

8. What is the use of hagrp command ?
Ans: hagrp is used for doing administrative actions on service groups like online, offline, switch etc.

9. How to switchover the service group in VCS ?
Ans: hagrp –switch -to

10. How to online the service groups in VCS ?
Ans: hagrp –online -sys

I thank all those who are contributing articles , interview questions, interview experiences, any other useful bit of information.

75 posts on Storage Jobs Interview Questions Veritas Solaris

This Blog has nearly 75 posts about Interview Questions, Check out Technorati's search results

Technorati Results of Storage Jobs Interview Questions Blog

Is there any shortcut for me to get High Salary Job

This is the most frequently asked question. Everyone wants to get a very high paying job. But people have very little time & efforts if I ask them to slog to achieve this. So to answer the question are there any shortcuts. Yes there are but it again requires you to put some more time & effort.

Without time & effort the only short cut I practically see is buying the job OR winning the job using heavy power influence.

Since most can not afford it let me try giving some guidelines to help achieve a better Career progress.

There are high demand job openings in both Storage Technology as well as Virtualization (Server Consolidation)
Companies like EMC2,Network Appliance,IBM,HP,Vmware,Accenture,Cisco,Intel,AMD almost all the giants we can remember everyone is getting heavily in these technologies. All of these companies are hiring & they are hiring not few but hunderds of Professionals.

This Blog has lots of info to help in this regard

1. Start with an area of your interest (Example : Testing domain or System Administration or Storage administration or Virtualization implementation etc)
2. Start knowing about what are the sub components in this area
3. See if you can get some course/training for the same - crash course do help,but more than that how many hours,years you put to gain practical experience
4. Start preparing your resume ( This helps you have a clear road map - many feel this is the hardest part to start but it helps if you build a brief resume & start developing it as you progress)
5. Parallely increase your knowledge base on specific key areas of your selective domain
6. Build contacts with people who can help you increase your exposure & experience (may be an experienced person can make the whole learning easier for you)
7. Get involved in projects,open source initiatives & contribute as much as you can (THIS IS THE ONE MOST CRUCIAL STEP)
8. Start building your Resume : Put Projects, Your Efforts, Your tasks, Role, Responsibilties
9. Be able to present in a mock presentation what you've learnt so far
10.Develop your ability to expand & work on extreme challenges - for this your learning & working experience helps a lot
11. Keep reading this blog ( As you can learn a lot of new things quite fast & quite easily)
12.Start preparing for mock interviews , ask a friend to take mock interviews & know where you stand
13.Start preparing for real interviews, Increase the effectiveness of your resume & make sure you are perfect in everything you are mentioning in your resume
14.Start applying to job portals. (I will write a post soon about how you can leverage this for better success)
15.Never loose heart no matter how long this process takes or how many interviews go without results- You can definitely get JOB :)

Array-based Flash Memory Could Enable 1TB Memory Chips

Array-Based Fash Memory

The alphabet soup of different flash memory technologies is already a little bewildering, but it looks like the latest entrant could end up being the most promising of all, with single chip storage capacities of 1TB expected within ten years. Called array-based memory, the tech has been under development at a company called Nanochip, Inc. for nearly 12 years, and it looks like the first working samples will go out next year. Although those first prototypes will have storage roughly equivalent to NAND flash at tens of gigs per circuit, the plan is to rapidly scale up to 100s of gigs and finally to 1TB on a single chip. Because the chips can be manufactured using conventional fabs and aren’t subject to the same manufacturing constraints as traditional flash, they may also end up being far cheaper per gigabyte. The company is being funded by a number of prominent tech giants, including Intel, and says the tech can be used to improve everything from USB keys to SSDs to enterprise-grade servers — wait, bigger, cheaper, and potentially better? Yeah, sign us up.

New Flash Drives From Super Talent Are Super Tiny

I’m a big fan of Super Talent’s USB drives. They’re tiny, weightless, high capacity, tiny, high quality, cheap, and tiny. The new versions are more of all that stuff, and one of them is billing itself as “the world’s smallest” 8 gig USB flash drive, weighing less than a nickel and measuring 31.3mm x 12.4mm x 3.4mm. There are three different versions; all of them are packin’ 8 gigabytes inside, but one has a swiveling cover, one has a retracting cover, and one (the tiny one) is completely naked. They can survive shocks and being submerged in water, come with a lifetime warranty, and only cost $35 each. The cheaper 4 gig versions are available now, here, but I’d hold out for the 8 gig version if I was you.

Storage area network connectivity overview

A storage area network (SAN) organizes a broad assortment of storage devices into a single storage resource that can then be provisioned, allocated and managed for the entire enterprise. Although issues like storage capacity, performance and management often receive the most attention, the connectivity between each SAN device plays a critical role in successful SAN deployment. Each switch and storage system on the SAN must be interconnected -- usually through optical fiber or copper cabling -- and the physical interconnections must support bandwidth levels that can adequately handle the peak data activities that occur. This overview details the role of Fibre Channel, Ethernet and iSCSI connectivity on a SAN.

Fibre Channel

Fibre Channel is the quintessential SAN interconnect and virtually every storage switch and storage platform provides Fibre Channel ports. Multiple Fibre Channel ports support simultaneous data streams, but individual ports can often be aggregated into groups for even higher effective bandwidth. As an example, the research guide notes that the InServ E800 Storage Server from 3PAR Data Inc. supports up to 128 Fibre Channel ports, while the TagmaStore AMS1000 from Hitachi Data Systems Inc. (HDS) provides up to eight Fibre Channel ports. Servers and other devices can also be fitted with Fibre Channel host channel adapters to enable an Fibre Channel interface.

As a serial interface, Fibre Channel bandwidth is denoted in Gbps. Early Fibre Channel implementations ran at 1 Gbps per port before doubling to 2 Gbps and then 4 Gbps. Another transition to 8 Gbps is beginning, but mainstream adoption is not expected until mid or late 2009. Also, 10 Gbps is used in interswitch links to connect Fibre Channel switches to each other. Fibre Channel operates with numerous protocols, most notably SAS and IP.

Fibre Channel can use several types of physical media. Twisted pair cable is used to cover relatively short distances at low speeds between Fibre Channel devices. Coaxial cables generally offer better shielding against signal interference and can run across somewhat longer distances. Optical fiber is routinely used to carry the fastest signals across distances up to 10 km.

Ethernet

While Ethernet connectivity is generally used on the greater local area network (LAN), its use in the storage area network has been limited by its relatively slow bandwidth. Traditional Ethernet ports support 10/100 Mbps -- far slower than Fibre Channel. This had limited Ethernet in the SAN to basic management tasks. For example, a storage device or switch might include a single Ethernet port that connects the device to the LAN where an administrator can manage the device across it. Ethernet typically uses two protocols; Transmission Control Protocol (TCP), which handles the organization of data into packets, and Internet Protocol (IP), which handles the way those data packets are addressed. In fact, the terms "Ethernet" and "TCP/IP" are often used interchangeably.

Ethernet bandwidth is increasing today, and 10 Gigabit Ethernet (10 GigE) is widely available for data centers, which boosts performance on the LAN and will eventually make Ethernet use more practical for carrying data on the SAN. One Gigabit Ethernet is now common on many servers and switches, and the eventual emergence of 10 GigE promises to put Ethernet on par with 10 Gbit Fibre Channel.

Traditional Ethernet LAN deployments used coaxial cables, but twisted-pair cabling (e.g., Category 5 or Category 6 Ethernet cables) is the most common LAN cabling. Ten GigE often relies on optical fiber with transmission distances up to 40 km, which makes the technology far more expensive and limits its use to network backbones. As copper cabling becomes available for 10 GigE, the technology should see far more use within data centers and storage area networks.

ISCSI and FCIP

Fibre Channel SANs have long been challenged by deployment expense and management complexity -- often keeping SANs out of reach of smaller IT organizations. The emergence of iSCSI eases these challenges by encapsulating SCSI commands into IP packets for transmission over an Ethernet connection, rather than a Fibre Channel connection. This approach eliminates Fibre Channel in favor of Ethernet, which allows iSCSI to transfer data over LANs, WANs or the Internet and supports storage management over long distances.

In actual practice, a user or application will cause the operating system to generate corresponding SCSI storage commands. Those SCSI commands and data are then encapsulated and IP headers are added to make packets. The packets can then be sent over an ordinary Ethernet connection. The remote end of the iSCSI connection disassembles the encapsulated content and passes the SCSI commands to the SCSI controller and storage device. This also works in reverse, so any data or responses can be sent back to the user or application across the Ethernet connection.

Although it is easier to manage than Fibre Channel, iSCSI still has two disadvantages for storage. At 1 GigE, it does not perform as fast as Fibre Channel. And Ethernet will drop packets during network congestion. These problems may be alleviated soon, thanks to the emergence of 10 GigE and Data Center Ethernet, a standard in development with the goal of providing Ethernet with quality of service levels, efficient multipathing and lower latencies while preventing dropped packets. Another alternative is FCIP. FCIP translates Fibre Channel commands and data into IP packets, which can be exchanged between distant Fibre Channel SANs. It's important to note that FCIP only works to connect Fibre Channel SANs, but iSCSI can run on any Ethernet network.

Storage vendors are working on an Fibre Channel over Ethernet (FCoE) standard to enable SAN and LAN convergence. FCoE would reduce cabling by using converged network adapters in place of Fibre Channel HBAs and NICs. FCoE adoption depends on the availability of Data Center Ethernet and is not expected to be widely adopted before late 2009. Unlike iSCSI, FCoE is not routable and is subject to the distance limitations of Fibre Channel.

Wednesday, April 9, 2008

History of Enterprise Disk to Disk Backup

In the early days of the 20th century it was not clear that the car would eventually replace the horse (and walking) as the most popular method of personal travel. Horses were cheaper, more familiar and more reliable than the early generation of petrol driven cars.

At the turn of the second millenium the tape backup market had reached a seemingly unassailable position. 3 out of the top 10 fastest growing profitable US storage companies in the first half of 2001 were tape companies:- ADIC (40% revenue growth), Overland (59% revenue growth) and Qualstar (40% revenue growth).

But the widespread corporate adoption of the internet which led to the dotcom bubble was also laying the connection paths for offsite backup and replication technologies which would soon cure the compulsive obsession of manually removing storage media offsite.

In 2006 the tape market saw its steepest ever decline in suppliers with mergers and acquisitions shrinking the supplier base. The death rattle of the tape backup market has been heard, although it may survive as a static niche market for a few more years.

This timeline plots the main markers for the decline of tape and the storage market switch to D2d.


1987 - the world's first NAS company Auspex Systems is founded. Their first servers were listed in our SPARC Product Directory in 1992. But it took the industry many years to realise that these products were a new type of storage server rather than platforms for databases and web servers.

1994 - STORAGEsearch.com's publisher publishes the first directory of fibre-channel adapters. That was the start of the network storage market which would later be called SANs. The market was hotly contested - and in 1996/97 there were far more oems making FC adapters than there are today. Many of them advertised in our online directory.

1997 - STORAGEsearch.com's publisher creates an online directory of RAID systems.

March 2000 - STORAGEsearch.com created a dedicated web page for online backup and storage. By this time the internet infrastructure, beefed up to host corporate websites for the dotcom economy was sufficiently well established to make internet backup technically viable. Before this era - this type of offsite online backup was restricted to big corporations with leased data lines using proprietary technology.

2000 - FalconStor Software is founded. The company would later become an important enabler for white box storage oems entering the virtual tape market.

August 2000 - An article called - the Return of Removable Hard Disk Drive Architecture - summarised the state of the art in SMB disk backup with a comparison of removable storage devices from leading vendors.

December 2000 - NAS climbs to equal place with SAN to reach #5 in popularity with STORAGEsearch.com's readers.

March 2001 - STORAGEsearch.com created a dedicated web page for iSCSI

September 3, 2001 - STORAGEsearch.com ran its first banner ad for a D2d product for Nexsan Technologies (see below).
click for more info
November 2001 - in an article called - the Next Decade in Storage - STORAGEsearch.com precisely foretold the reasons why D2d would eventually replace tape backup.

March 2002 - STORAGEsearch.com created a dedicated web page for D2d

May 2002 - ads on STORAGEsearch.com's main D2d page were split 50:50 between new D2d systems and tape libraries as you can see on the archived page.

December 2002 - in STORAGEsearch.com's year end review - SATA was named Product Category of the Year - based on reader pageviews. SATA hard drives provided a step change in capacity and system cost which later helped to accelerate the D2d market.

February 2003 - the first recorded occurrence of the term VTL (virtual tape library) appears in a joint news story by MTI Technology and Quantum.

April 2003 - STORAGEsearch.com reported on May 5, 2003 that reader pageviews for disk to disk backup in April 2003 exceeded pageviews for tape backup for the first time. "We're now at a watershed where users no longer have to choose tape just because their forebears did" - said editor Zsolt Kerekes at the time.

October 2003 - StoneFly Networks and CommVaultSystems launched a bundled D2D backup IP SAN system that starts at $33,995 for a 1TB solution.

November 2003 - commenting on the launch of its new ATAbaby disk array Nexsan's Senior Executive Vice President Diamond Lauffin said "The new ATAbaby Twins is another nail in the coffin of tape-based backup. No tape loader or library can deliver the same level of data protection and availability at anywhere close to this price."

January 2004 - COPAN Systems summarised the current state of the tape - verses D2d market. "Unfortunately, even with the advent of the new SATA disk systems at $10/GB, disk is still three to five times the cost of automated tape libraries making long term data management prohibitive. In addition, many enterprises are concerned with the reliability of SATA technologies."

August 2004 - Peripheral Concepts published the results of their market research into backup practises and plans in over 1,000 major IT sites. The report said:- "The population using disk in backup has grown to 62%, and is forecast to reach 76% penetration by 2005."

December 2004 - Engenio published an article called - Disk to Disk Backup versus Tape - War or Truce?

Commenting on this STORAGEsearch.com said "Will disk to disk backup make tape backup obsolete? That's a question that's been debated hotly here on STORAGEsearch for many years. At the extreme polarized ends of the argument are tape media makers like Sony, who in an article here made a case for the long term survival of tape, and at the other end of the argument are disk to disk supporters like STORAGEsearch whose editorial view (March 2004) has been that tape doesn't have a viable role the midsize market any more.

March 2005 - Quantum became the first major storage vendor to include data compression within a D2d product when it added its Optyon hardware based data compression technology to the DX-Series disk-based backup systems enabling 2T / hour backup performance. In addition to the cost savings compared to uncompressed solutions - an additional benefit is increased storage reliability for customers. Because they only need half as many disks in their disk backup population.

Overland claimed to be the #1 company in disk to disk backup, saying it had shipped more disk-based backup appliances than any other vendor:- more than 1,000 of its own REOs. I was sceptical about this leadership claim - but no one contested it at the time.

October 2005 - MaXXan Systems published an article called - Virtual Tape: Can You Afford to Ignore It?

Commenting on this STORAGEsearch.com said "Some of the growing sophistication in the market can be seen by the way that the marketing terminology has morphed from the early D2d (let's kill tape backup) to the current VTL (Virtual Tape Library - let's just see if they notice that it's more reliable and works faster - and don't tell them that there isn't a tape in the box) type of approaches."

May 2006 - in a white paper called "Total Data Protection" - Farid Neema, President of Peripheral Concepts wrote - "For the first time ever, more than one half of the surveyed managers consider the possibility of a tapeless IT operation in the long run. This aspiration for "getting rid" of tapes is even more pronounced in very large IT operations. In 2006, there will be more data backed up on disk than on tape, and disk should gain 4% share every year for the next three years."

August 2006 - D2d became the 4th most popular subject viewed by STORAGEsearch.com's readers (up from #15 in July). I commented - "Recent months have seen a spate of mergers and revenue attrition in the tape backup market - and the sharp increase in D2d content may signal that users no longer give credence to all the brave bullish talk they've been hearing from tape backup companies.

September 2006 - D2d became the 2nd most popular subject viewed by STORAGEsearch.com's readers.

October 2006 - D2d became the #1 most popular subject viewed by STORAGEsearch.com's readers. ...later:- and it remained #1 for the next 8 months too - upto and including May 2007.

Also in October 2006 - Network Appliance added data compression to its Virtual Tape Libraries.

December 2006 - D2D was named STORAGEsearch.com's Product Category of the Year 2006. In Q406 there was a surge of interest in D2D - which accounted for more reader pageviews than the next two subjects added together, despite annual pageview growth over 50% in those subjects too. "These results predict a stunning growth year for the disk to disk backup industry in 2007" said Zsolt Kerekes, editor STORAGEsearch.com.

January 2007 - Hitachi announced the industry's first terabyte hard drive priced at about 40 cents per gigabyte.

March 2007 - Enterprise Strategy Group quoted from their VTL Adoption and Market Trends Report that "85% of VTL users have already eliminated tape from their daily backup schedules."

April 2007 - STORAGEsearch.com reported that tape backup related searches had dropped out of the top 20 subjects viewed by readers in the preceding month.

June 2007 - Idealstor announced it had certified Hitachi's new terabyte hard disks for use with its Teralyte removable disk backup system.

November 2007 - NetApp's CEO Dave Hitz said in his blog this month "...I'm surprised how important creating real tapes from the VTL has remained. Despite all the hoopla about disk-to-disk backups, 80% of VTL customers still rely on tape for some part of their process."

December 2007 - Disk to Disk Backup is named STORAGEsearch.com's Product Category of the Year 2007 - for the 2nd year in a row.

Infiniband FAQs

What is InfiniBand Architecture?
InfiniBand Architecture is an industry standard, channel-based, switched fabric, interconnect architecture for servers. InfiniBand architecture changes the way servers are built, deployed, and managed.

Why is InfiniBand Architecture important?
As processor speeds increase with Moore's Law and the Internet drives the demand for constantly available data, the biggest gate to improved data center performance is the I/O subsystem. InfiniBand Architecture provides a blueprint for significantly improved solution performance and increased availability to meet the needs of the Internet. InfiniBand Architecture is designed to fulfill the need for greater data center reliability, availability and scalability, as well as greater design density for InfiniBand architecture enabled servers.

What performance range is offered by the InfiniBand Architecture specifications?
InfiniBand Architecture offers three levels of link performance - 2.5 Gbits, 10 Gbits, and 30 Gbits/sec. InfiniBand Architecture also enables low latency communication within the fabric enabling higher aggregate throughput than traditional standards-based protocols. This uniquely positions InfiniBand Architecture as the I/O interconnect for data centers.

What will it take to integrate InfiniBand Architecture into a data center?
InfiniBand Architecture is expected to be implemented with existing data center implementations through connectivity into the InfiniBand fabric. Initial server implementations are expected to include InfiniBand Host Channel Adapters deployed on PCI adapter cards. As the technology evolves, native InfiniBand implementations are expected with a broad base of interoperable products from member companies. IT managers should talk to their suppliers about migration strategies to InfiniBand Architecture enabled solutions.

What is the expected demand for InfiniBand Architecture enabled solutions?
InfiniBand Architecture solutions create new opportunities for product development. According to IDC, initial InfiniBand architecture enabled servers are expected to ship in the first half of 2002 with 50% of total servers enabled with InfiniBand architecture by 2005.

What specific benefits do IT managers anticipate with InfiniBand fabric rollout?
IBTA IT program members have identified many areas of interest in InfiniBand fabric deployment including increased I/O performance for leading applications including database clusters, enhanced server density, and multi-vendor system interoperability.

How will InfiniBand fabrics be managed?
The InfiniBand Architecture specifications have standardized InfiniBand management infrastructure. InfiniBand fabrics will be managed via InfiniBand consoles and InfiniBand fabric management is expected to snap into existing enterprise management solutions.

How big is the InfiniBand Architecture development effort?
The InfiniBand Trade Association has grown from 7 companies to more than 190 since its launch in August, 1999. Membership is open to any company, department of government or academic institution interested in the development of InfiniBand architecture. To see a list of current trade association members please visit the roster.

When are products based on the InfiniBand Architecture Specification expected in the marketplace?
Initial InfiniBand fabric implementations are expected to appear in 2002. As with many new data center technologies, customers are expected to test InfiniBand architecture enabled solutions through pilot programs prior to mass rollout of the technology. To read about how some data centers are planning on using InfiniBand architecture visit www.infinibandta.org/itinfo/it.

Over sixty member companies have already announced plans for products based on InfiniBand Architecture. To see a list of member company InfiniBand architecture-related announcements visit www.infinibandta.org/newsroom/articles.

How does InfiniBand Architecture relate to 3rd Generation I/O technology (3GIO)?
The technologies are complimentary. InfiniBand Architecture was designed from the ground up to address the specific needs of data centers. In the server market, 3GIO is expected to be an inside the box interconnect, or local I/O.

What is the relationship between InfiniBand and Fibre Channel or Gigabit Ethernet?
InfiniBand Architecture is complementary to Fibre Channel and Gigabit Ethernet. InfiniBand architecture is uniquely positioned to become the I/O interconnect of choice for data center implementations. Networks such as Ethernet and Fibre Channel are expected to connect into the edge of the InfiniBand fabric and benefit from better access to InfiniBand architecture enabled compute resources. This will enable IT managers to better balance I/O and processing resources within an InfiniBand fabric.

How does InfiniBand Architecture relate to Rapid I/O?
Rapid I/O is one of many industry I/O technologies focused at embedded applications. Although it is possible to use InfiniBand Architecture in embedded designs, we believe the technologies have different focuses.

What type of cabling will the new I/O architecture support?
In addition to a board form factor connection, it supports both copper (up to 10 meters or longer) and fiber-optic cabling (up to 10km).

How many nodes will InfiniBand Architecture support?
InfiniBand Architecture is capable of supporting tens of thousands of nodes in a single subnet.

NAS, DAS or SAN? - Choosing the Right Storage Technology for Your Organization

While one type of storage media is usually sufficient for smaller companies, large enterprises will often have a mixed storage environment, implementing different mediums for specific departments, workgroups and remote offices. In this paper, we will provide an overview of DAS, NAS and SAN to help you determine which solution, or combination of solutions, will best help you achieve your business goals.

DAS: Ideal for Local Data Sharing Requirements

Direct-attached storage, or DAS, is the most basic level of storage, in which storage devices are part of the host computer, as with drives, or directly connected to a single server, as with RAID arrays or tape libraries. Network workstations must therefore access the server in order to connect to the storage device. This is in contrast to networked storage such as NAS and SAN, which are connected to workstations and servers over a network. As the first widely popular storage model, DAS products still comprise a large majority of the installed base of storage systems in today's IT infrastructures.
low  cost - 3U 12 bay USB / FireWire / SATA / Infiniband  storage
3U 12 bay Infiniband storage
from Norco Technologies
Although the implementation of networked storage is growing at a faster rate than that of direct-attached storage, it is still a viable option by virtue of being simple to deploy and having a lower initial cost when compared to networked storage. When considering DAS, it is important to know what your data availability requirements are. In order for clients on the network to access the storage device in the DAS model, they must be able to access the server it is connected to. If the server is down or experiencing problems, it will have a direct impact on users' ability to store and access data. In addition to storing and retrieving files, the server also bears the load of processing applications such as e-mail and databases. Network bottlenecks and slowdowns in data availability may occur as server bandwidth is consumed by applications, especially if there is a lot of data being shared from workstation to workstation.

DAS is ideal for localized file sharing in environments with a single server or a few servers - for example, small businesses or departments and workgroups that do not need to share information over long distances or across an enterprise. Small companies traditionally utilize DAS for file serving and e-mail, while larger enterprises may leverage DAS in a mixed storage environment that likely includes NAS and SAN. DAS also offers ease of management and administration in this scenario, since it can be managed using the network operating system of the attached server. However, management complexity can escalate quickly with the addition of new servers, since storage for each server must be administered separately.

From an economical perspective, the initial investment in direct-attached storage is cheaper. This is a great benefit for IT managers faced with shrinking budgets, who can quickly add storage capacity without the planning, expense, and greater complexity involved with networked storage. DAS can also serve as an interim solution for those planning to migrate to networked storage in the future. For organizations that anticipate rapid data growth, it is important to keep in mind that DAS is limited in its scalability. From both a cost efficiency and administration perspective, networked storage models are much more suited to high scalability requirements.

Organizations that do eventually transition to networked storage can protect their investment in legacy DAS. One option is to place it on the network via bridge devices, which allows current storage resources to be used in a networked infrastructure without incurring the immediate costs of networked storage. Once the transition is made, DAS can still be used locally to store less critical data.
serial attached scsi chassis from Enlight 1u to 5u
Serial Attached SCSI Chassis 1U to 5U
from Terabytes Server Storage Tech
NAS: File-Level Data Sharing Across the Enterprise

Networked storage was developed to address the challenges inherent in a server- based infrastructure such as direct-attached storage. Network-attached storage, or NAS, is a special purpose device, comprised of both hard disks and management software, which is 100% dedicated to serving files over a network. As discussed earlier, a server has the dual functions of file sharing and application serving in the DAS model, potentially causing network slowdowns. NAS relieves the server of storage and file serving responsibilities, and provides a lot more flexibility in data access by virtue of being independent.

NAS is an ideal choice for organizations looking for a simple and cost-effective way to achieve fast data access for multiple clients at the file level. Implementers of NAS benefit from performance and productivity gains. First popularized as an entry-level or midrange solution, NAS still has its largest install base in the small to medium sized business sector. Yet the hallmarks of NAS - simplicity and value - are equally applicable for the enterprise market. Smaller companies find NAS to be a plug and play solution that is easy to install, deploy and manage, with or without IT staff at hand. Thanks to advances in disk drive technology, they also benefit from a lower cost of entry.

In recent years, NAS has developed more sophisticated functionality, leading to its growing adoption in enterprise departments and workgroups. It is not uncommon for NAS to go head to head with storage area networks in the purchasing decision, or become part of a NAS/SAN convergence scheme. High reliability features such as RAID and hot swappable drives and components are standard even in lower end NAS systems, while midrange offerings provide enterprise data protection features such as replication and mirroring for business continuance. NAS also makes sense for enterprises looking to consolidate their direct-attached storage resources for better utilization. Since resources cannot be shared beyond a single server in DAS, systems may be using as little as half of their full capacity. With NAS, the utilization rate is high since storage is shared across multiple servers.

The perception of value in enterprise IT infrastructures has also shifted over the years. A business and ROI case must be made to justify technology investments. Considering the downsizing of IT budgets in recent years, this is no easy task. NAS is an attractive investment that provides tremendous value, considering that the main alternatives are adding new servers, which is an expensive proposition, or expanding the capacity of existing servers, a long and arduous process that is usually more trouble than it's worth. NAS systems can provide many terabytes of storage in high density form factors, making efficient use of data center space. As the volume of digital information continues to grow, organizations with high scalability requirements will find it much more cost-effective to expand upon NAS than DAS. Multiple NAS systems can also be centrally managed, conserving time and resources.

Another important consideration for a medium sized business or large enterprise is heterogeneous data sharing. With DAS, each server is running its own operating platform, so there is no common storage in an environment that may include a mix of Windows, Mac and Linux workstations. NAS systems can integrate into any environment and serve files across all operating platforms. On the network, a NAS system appears like a native file server to each of its different clients. That means that files are saved on the NAS system, as well as retrieved from the NAS system, in their native file formats. NAS is also based on industry standard network protocols such as TCP/IP, FC and CIFS.
Axstor for iSCSI NAS  storage
Axstor, based in Coventry in the UK, provides
world class high-performance iSCSI storage
solutions at a cost-effective price point.
SANs: High Availability for Block-Level Data Transfer

A storage area network, or SAN, is a dedicated, high performance storage network that transfers data between servers and storage devices, separate from the local area network. With their high degree of sophistication, management complexity and cost, SANs are traditionally implemented for mission-critical applications in the enterprise space. In a SAN infrastructure, storage devices such as NAS, DAS, RAID arrays or tape libraries are connected to servers using Fibre Channel. Fibre Channel is a highly reliable, gigabit interconnect technology that enables simultaneous communication among workstations, mainframes, servers, data storage systems and other peripherals. Without the distance and bandwidth limitations of SCSI, Fibre Channel is ideal for moving large volumes of data across long distances quickly and reliably.

In contrast to DAS or NAS, which is optimized for data sharing at the file level, the strength of SANs lies in its ability to move large blocks of data. This is especially important for bandwidth-intensive applications such as database, imaging and transaction processing. The distributed architecture of a SAN also enables it to offer higher levels of performance and availability than any other storage medium today. By dynamically balancing loads across the network, SANs provide fast data transfer while reducing I/O latency and server workload. The benefit is that large numbers of users can simultaneously access data without creating bottlenecks on the local area network and servers.

SANs are the best way to ensure predictable performance and 24x7 data availability and reliability. The importance of this is obvious for companies that conduct business on the web and require high volume transaction processing. Another example would be contractors that are bound to service-level agreements (SLAs) and must maintain certain performance levels when delivering IT services. SANs have built in a wide variety of failover and fault tolerance features to ensure maximum uptime. They also offer excellent scalability for large enterprises that anticipate significant growth in information storage requirements. And unlike direct-attached storage, excess capacity in SANs can be pooled, resulting in a very high utilization of resources.
Hard Drive Degaussers sanitize disks  even when they can't be erased using software
Hard Drive Degaussers & Destroyers
from Storage Heaven
There has been much debate in recent times about choosing SAN or NAS in the purchasing decision, but the truth is that the two technologies can prove quite complementary. Today, SANs are increasingly implemented in conjunction with NAS. With SAN/NAS convergence, companies can consolidate block-level and file-level data on common arrays.

Even with all the benefits of SANs, several factors have slowed their adoption, including cost, management complexity and a lack of standardization. The backbone of a SAN is management software. A large investment is required to design, develop and deploy a SAN, which has limited its market to the enterprise space. A majority of the costs can be attributed to software, considering the complexity that is required to manage such a wide scope of devices. Additionally, a lack of standardization has resulted in interoperability concerns, where products from different hardware and software vendors may not work together as needed. Potential SAN customers are rightfully concerned about investment protection and many may choose to wait until standards become defined.

Conclusion

With such a variety of information storage technologies available, what is the best way to determine which one is right for your organization? DAS, NAS and SAN all offer tremendous benefits, but each is best suited for a particular environment. Consider the nature of your data and applications. How critical and processing-intensive are they? What are your minimum acceptable levels of performance and availability? Is your information sharing environment localized, or must data be distributed across the enterprise? IT professionals must make a comprehensive assessment of current requirements while also keeping long-term business goals in mind.

Like all industries, storage networking is in a constant state of change. It's easy to fall into the trap of choosing the emerging or disruptive storage technology at the time. But the best chance for success comes with choosing a solution that is cost-correct and provides long term investment protection for your organization.