Popular transaction processing oltp software in ibm mainframes

Introduction to z/OS Concepts and Processing Modes

Introduction to z/OS

IBM z/OS is the flag ship zEnterprise operating system of the zEnterprise Mainframe platform. The original IBM operating systems, e.g., OS 360, OS390, etc. were designed for batch processing, had limited Internet application capabilities and connectivity. The IBM mainframe has always been the IT industry's leading platform for transaction processing, consolidated and secure data services and storage, and for supporting enterprise-wide applications. There were NO alternative substitutes for IBM mainframes and operating systems.

In the late 2001, zEnterprise introduced the 64-bit z/OS operating system that extended their dominance of the large-scale transaction processing market place. z/OS supports TCP/IP, any Internet application subsystems, e.g., HTTP, FTP, SSH, etc., most major programming languages, e.g., Java, C++, etc., data bases. e.g., Oracle, MySQL, etc. and also directly supports and integrates UNIX and the UNIX file system.

All new zEnterprise features are fully backward compatible to the 50-year legacy heritage of the mainframe. Fedora Linux guarantees backward-compatibility for two years; RedHat Enterprise Linux guarantees backward compatibility for seven years. Backwards compatibility varies considerably for MicroSoft's operating systems, programming languages and applications. For example, Visual Basic applications of the 1990s are not compatible with VB.NET or newer Windows operating systems. Versions of SQL Server databases introduced in the last 10 years are backward compatible but earlier versions are not (but can be converted to a new version). zLinux can execute open source software, but Windows cannot.

There is no other architecture or operating system that provides the flexibility, backward compatibility and integrates all of functions as well as z/OS. Backward compatibility means that the organization never loses their investment in applications, system configurations, and data. But, zEnterprise and z/OS may not be cost-effective at lower level of scales of transaction processing and supporting desktops.

The following diagram displays a summarized view of z/OS operating system and subsystems. At the top of the diagram is the host operating system, i.e., z/OS, The subsystems and interfaces listed in the far right columns are normally used by application and system programmers. The subsystems and interfaces listed in the far left column are normally used by customers, users, and other programs. The subsystems and interfaces list in the middle column provide additionally functionality to z/OS in areas such as database management, security and communications.

The z/OS operating system is a collection of base control modules (programs) that provide functions similar to any other operating system, e.g., MS-DOS, Windows, Linux, Mac OS, Android, etc. There are two differences between z/OS and the previously listed operating systems.

First, over the last 50 years IBM operating systems introduced concepts that we take for granted in Windows, Unix and Linux. These operating systems have modified technologies that was first developed by IBM mainframes and operating systems decades previously, MS-DOS and Windows 1.0 were introduced in 1980s thirty years after IBM had performed many of the same tasks. MS-DOS appeared 25 years after IBM started their dominance. [A history of Windows - http://windows.microsoft.com/en-us/windows/history#T1=era0]

Consider the following - Who invented virtual memory operating systems and file systems? Answer IBM. Other operating systems copied virtual memory concepts and introduced their modified version s of virtual memory operating systems 30 years later. Who invented System and Server Virtualization? VMware - no. IBM. And the list goes on. The majority of concepts that an IS or CS major learns in a class started with IBM decades previously.

IBM is no longer the #1 IT company. Apple is. Why? Apple is a consumer-based IT company which is much more interesting and fun and with a larger potential sales market. IBM is a very large-scale, business transaction processing company directing its services to large businesses, government agencies and other organizations. IBM's target market is not address to consumer technologies.

Second, zEnterprise is the only viable secure, large-scale transaction processing system available. Did you "text" today? Yes, I did I text on my Verizon phone. O.K. What platform does Verizon use to process and route text messages? 95% of all text messages are processed by an IBM mainframe. One-third of the world wealth is processed and stored on an IBM mainframe. 71% of all banking, insurance and financial service transactions worldwide are processed by an IBM mainframe, 51% of retailing and 46% of government transactions.

Do these companies and government agencies love IBM? Hell, no. There is no alternative. The past and the future disasters of the O' Bama Health Care website are proof of the limitations of medium-scale, microprocessor-based, distributed systems.

First, let's determine the functionality of the OBC website. The OBC website simply adds (inserts) individuals to the ACA (Affordable Care Act) database based on some parameters and then routes the registered ACA user to sign-up for health care at a third-party web site. The OBC website does not even process an insurance sign-up transaction. It doesn't even perform even the simplest transactions we are accustomed to on the Amazon.com website. [Which Enterprise architecture does Amazon use? Do you even have to ask?]

Second, the reporting transaction processing bottle neck at OBC the web site is that it cannot support more than 300,000 logon users and transactions per day. [Can you imagine Amazon with this limitation? JP Morgan Chase is the home of VISA credit card transaction processing. One zEnterprise EC12 mainframe running z/OS at JP Morgan Chase normally processes 3 million logon transactions securely and reliably each minute. NO zEnterprise system has ever been externally hacked or data was compromised. Target processes it Visa credit cards on this system. Target's card swipers were compromised at the point of data-entry, not the zEnterprise system.

Table 1 summarizes the core operating system function provided by z/OS and most modern operating systems. Table 2 summarizes the functions of popular z/OS subsystems.

Table 1

z/OS Core Functions

Description and Comparison

Task and Time Management

An operating system task, or processes, is an executing program under the control of the operating system. In order to execute the task must be allocate processor time. An operating system will execute these tasks using different processing modes (workloads), e.g., batch, time sharing, etc. In a multitasking operating system, one must trade off efficiency (throughput) with response time. Batch processing is very efficient, but has poor response time when multi-users are waiting. (See Processing Modes later.)

Memory Management

All programs and data must be loaded into physical memory to be executed. Physical memory is very fast and very expensive. The total physical memory size is limited. Most modern operating system using virtual memory. A VM OS will temporarily store pages of executing programs and data on a disk waiting for execution, and then swapped back to main memory as needed. VM OS have the ability to manage the execution of large amounts of tasks and data by using an auxiliary storage device as a storage overflow.

I/O Control

Auxiliary storage systems, network devices and other computer peripherals are connected to CPU and system memory. OS software must be used to control the communication between these components. Windows uses the concept of "I/O drivers" to describe I/O Control software.

In very early versions of IBM OS, the I/O control and communication programs where compiled into the operating systems. This compilation method provided the maximum efficiency for I/O communication, but made it difficult to modify the I/O Configuration. Legacy IBM I/O control was based on device type, channels and unit numbers.

In recent versions IBM support popular microcomputer I/O interfaces (PCIe) and the flexible I/O driver model.

File Systems

A files system is a collection of modules that controls how data is stored and retrieved in a meaningful way. The disk I/O control system will store data in the format of cylinders, tracks, and perhaps sectors (Windows, Linux and Unix use sectors). But, that format is not meaningful and understandable to normal users.

The basic unit of storages for the z/FS is the volume and data set. A data set is similar to file in Windows. The content of a data set is very primitive as compared to Windows files. The content (type) of a Windows file normally is specified by its file extension, e.g., *.DOCX, MP4, *.MDBx, *.HT ML , etc.

The methods used by a file system to organize files vary considerably. The Windows NTFSv5 file system is a hierarchical directory structure which uses subdirectories or folders. z/FS uses a VTOC table which is similar to the root directory of NTFSv5, uses a flat file system and a Partitioned Data Set (PDS) to organize data set that less similar to a subdirectory or folder. (See Introduction to File Systems.)

File System Integration

The z/FS was made for efficient file storage and processing, not flexibility. The Windows NTFSv5 and Linux ext-4 file systems were designed for the flexibility of modern desktop and mobile operating system. z/OS supports and integrates UNIX and the UNIX file systems. The beauty of zEnterprise is that one view and process data sets and files by using either the z/FS for legacy applications and the Unix file systems on the same storage device.

Review of Major z/OS Subsystems (The follow graphic has been repeated for your convenience.)

Table 2

Major z/OS Subsystems

Description and Comparison

TSO/E

TSO is both a user interface and a processing mode. The original IBM operating systems were limited to punched cards. There were no user-friendly interfaces and remote processing was very difficult.

TSO (Time Sharing Option) provides a command-line interface similar to the command prompt (command line interface) in Linux and Windows. In addition TSO provides timesharing services to improve the response time for system and application programmers. TSO is normally used by system administrators and developers, not everyday users.

TSO, like MS-DOS and Linux CLIs, requires one to memorize and type interactive commands and parameters. (A command parameter is input information appended to a command to customize the execution of the command.)

ISPF

ISPF is an alternative command interface to TSO that displays fill-in the blanks menu panels to execute TSO commands. TSO and ISPF perform similar functions, except TSO commands are more flexible and powerful. (All command line interfaces (CLI) are more powerful than GUI interfaces.) A mobile version of ISPF is available for your cell phone on Google.

ISPF also provides a full screen text editor. TSO only provides for a line editor. The ISPF editor was designed for program development efficiency, but does take a little effort for Word users to get accustom to it.

JES3

JES (Job Entry System) receives submitted jobs and schedules these jobs for processing by z/OS. JES uses and processes JCL (Job Control Language) commands to control this job-entry process. Submitting a Job to JES is similar to starting a program execution in Windows by clicking on an Icon. When a program is started in Windows, detailed commands and parameters to control the process are stored in the Window's registry. In IBM, the JCL commands are very explicit and never hidden.

Table 2 ( continued)

Major z/OS Subsystems

Description and Comparison

SDSF

SDSF (System Display and Search Facility) allows users and administrators to view and control various aspects of the mainframe's operation and system resources. Batch job output, status information of running processes, system information, workload scheduling, external device monitoring such as printers and initiators. SDSF may also be used to access the batch and system log files and dumps.

SDSF is used by application programmers to review JCL, syntax and run-time errors to debug programs. Batch application programs can review reports and other customized debugging outputs. System programmers can start, stop and determine the status of other z/OS subsystems, like those listed in this table.

CICS

CICS (Customer Information Control System) was introduced in 1965 almost 30 years before web applications became available. Batch processing has no user interface and was designed to process large amounts of input transaction data. With CICS users may enter data for one transaction at a time (See OLTP below.)

Traditionally, CICS had used an interface called a mapset, which was very similar to a HTML document. 25 years later HTML documents copied the concepts introduced by mapsets and applied these to TCP/IP. Today, CICS can enable COBOL applications to use HTML document and web service interfaces. Java applications may also execute more efficiently under CICS as compared to its traditional J2EE application server.

CICS executes transactions. A transaction may represent the purchase of one or more products at Amazon.com or a deposit in a commercial bank. A transaction is a unit of work. The mapset or web page is the user interface to input or output this unit of work. It is better to view CICS as a super transaction processing server, rather than a simple user interface. In fact, there are three times as many CICS transaction processed each day as the total number of web pages served each day. Review this video CICS - Facts You Might Not Know - https://www.youtube.com/watch?v=fbGmlrKH8Aw

z/OS HTTP Web Server

A modified version of the Apache web server is used by z/OS to support static web pages. The basic Apace HTTP configuration is identical to Apache versions installed on Linux, Windows or Unix. However, z/OS Apache has been optimized to improve performance by leveraging the unique advantages of z/OS.

HTTP web servers are designed to serve static web pages, rather than dynamic web pages A static web page is a file that never changes. Static web pages cannot support transactions or Ecommerce web applications. A web application server, e.g., WebSphere or Tomcat, and be added or connected to a HTTP server to process dynamic web pages.

WebSphere

WebSphere is IBM proprietary J2EE application server, which support Java Server Pages, Java Server Faces, Java Servlets and Java Enterprise Beans applications. This is the technology that is used by many large-scale eCommerce providers, e.g., Blackboard, Barnes and Noble, Sam's Club, Walmart, etc. However, WebSphere leverages the unique scalability, efficiency and security found in a zEnterprise system not found in other Java alternatives. WebSphere is a User-Application system.

While WebSphere is a popular J2EE system used to drive dynamic web page application systems, CICS COBOL web applications execute almost twice as fast as traditional J2EE. But just like it is difficult to replace COBOL, it will be difficult to replace J2EE applications.

Table 2 ( continued)

Major z/OS Subsystems

Description and Comparison

WebSphere MQ

WebSphere and J2EE are based on user interaction between a web application and an Internet User. WebSphere MQ (Message Queuing) uses web services that permits one application to interact with another remote application to process OLTP transactions with user interaction.

For example, suppose that one submits a request to your broker's web site to purchase 500 shares of IBM stock. This user-entry transaction request will start an automated sequence of transactions (called messages) between10 to 20 different companies' applications (called services). Web Service transaction processing and notification systems require little or no user interaction. Over 50% of new business application development requirements are based on web services or message queuing. MQ is an Application-to-Application system.

DB2

zFS datasets do not provide multiuser capabilities or features other than simple data storage. DB2 is the original relational database system that introduced SQL. Besides storing data, DB2 tables provide multi-user capabilities (concurrency), program language independence, enhanced security, referential integrity, transaction integrity, etc..

Oracle is a competing data base management system (DBMS), which provides most features similar to DB2. Large-scale Oracle DBMS systems are almost always installed on an IBM mainframe using z/OS. Oracle does provide more bells-and-whistles, ease-of-use, support for more operating systems, and flexibility than DB2. However, no DBMS processes large scale database transactions faster than DB2. DB2 is also available for zLinux.

IMS

IMS (Information Management System) was the first large-scale commercially available database product developed in 1966. IMS was a hierarchical database system that was originally designed for batch processing applications. In the 1980s, IMS represented 81% of the database management systems in use. Today that number has fallen to below 10%.

RACF

RACF (Resource Access Control Facility) is a z/OS integrated security server. Top Secret is a CA Technologies (an IBM third party provider) provides for a competing security server product.

Most operating systems, including z/OS, provide some type of security such as authentication services (User Accounts and Passwords) and authorization services (file system and operating system permissions). A Security Server, e.g., RACF and Top Secret, greatly extends and centralizes all security requirements. For example, DB2, IMS, WebSphere, and CICS have separate sophisticated security services relevant to its functionality. RACF permits DB2, IMS, CICS, WebSphere and z/OS to share the same user account and provide single logon, yet applying and centralizing the security policy appropriate to each subsystem. RACF also provides network security applications like firewall, intruder detection, etc., which may be integrated across z/OS subsystems.

z/OS Communication Server

SNA

TCP/IP was introduced to the military in 1982. In the early 1990s universities and the public started using TCP/IP. In 1974 IBM introduced its proprietary communication protocol called SNA (System Network Architecture). SNA provided a secured, long-distance communication protocol for business long before the TCP/IP became popular. Due to its security and performance strengths, SNA still continues to be used. The z/OS Communication Server supports and integrates TCP/IP and SNA services and protocols.

TCP/IP identifies network resources by IP Address; SNA identifies network resource by physical and logical unit numbers.

zEnterprise FTP Server

and

SSH Server

Permits transfers of programs, installation packages and data between remote clients, e.g., your computer or cell phone, and the z/OS and z/Unix.

The SSH server will provide secure terminal access z/UNIX using clients such as Putty. While ISPF cannot be accessed by Putty, most z/OS TSO functions are available through Putty.

Introduction to z/OS Workloads or Processing Modes

IBM supports four types of processing modes to support application and system programs: batch processing, CICS, WebSphere, and WebSphere MQ. IBM refers to each types of processing modes as "Work Loads".

Batch Processing

The original and most efficient processing mode is called Batch Processing. After an application has been developed, a batch application is submitted to the Job Entry System (JES3) using JCL for immediate or delayed (hold queue) execution. Once the JCL is processed by JES, it will review the availability of required resources, e.g., processing time, RAM, input and output data sets, etc., before the job is actually started.

While many consumer applications have shifted to web and mobile applications that process real time transactions, the use of batch processing continues to evolve. Local, state and the federal government (especially the IRS and Social Security administration) use batch processing to processes taxes, benefits, and billing. While payroll systems may enter data online, payroll processing applications are normally executed using batch processing. Modern data warehouses transfer transaction data from transaction databases using a batch procedure called extraction, transfer and load (ETL), and data mining and other computation processing relies on batch processing.

Advantages of Batch Processing

· Requires no user interaction; hence does not wait for user input of data or initiation of job execution. Thus, it avoids idling of computing resources with minute-by-minute human interaction and supervision.

· Batch jobs can be scheduled to take advantage of idle time and load balancing requirements.

· Minimizes idle time of required resources, e.g., input data sets, waiting output devices. High utilization rate.

· With proper scheduling it permits resources to be shared more efficiently, assign priorities, and with less conflicts.

· Generally preferred for heavy computational processing, significant update transaction workloads, long running jobs to be executed on multiple nodes (clusters or grid computing), e.g., IBM SysPlex and Parallel SysPlex infrastructure

Disadvantages of Batch Processing

· No uses interaction or support for Online Transaction Processing (OLTP), e.g., transaction and updates entered by users are processed immediate.

· No web or mobile interfaces.

· Limited sharing of datasets

Customer Information Control System (CICS)

Online Transaction Processing (OLTP) is a processing mode that users normally enter data for a small transaction and the system is updated immediately. For example, a batch program may calculate interest charge or update payments for all customer accounts based on inputs from a dataset. But, OLTP will permit users and employees to enter data on form, one-customer-at-time, and then immediately update the customer's balance one-customer at a time.

IBM CICS, a high volume OLTP system, was introduced in 1965. For decades, there were not viable commercial alternatives. OLTP and CICS systems have two major components: an online transaction monitor and an application programming interface (API).

Any operating system has very limited capabilities to share applications and data between users. For example, Windows XP introduced multithreading in Microsoft's desktop operating systems. Multithreading is the ability of an operating system to execute and share the same copy of an executable module, e.g., Internet Explorer, using the same process id (task number). Multitasking can execute two or more separate copies of an executable module using a separate task number and duplication of resources. Batch processing uses multitasking, but more important cannot concurrently share data sets.

CICS,a transaction monitor, provides multithreading services and the ability to share VSAM datasets in a read-write mode. DB2 and IMS will provide the abilities to share tables, but not VSAM data sets. CICS may be compared to modern web application servers, e.g., Apache Tomcat (free open source) and Web Sphere, Apache Geromino, .NET framework, JBOSS (proprietary) All provide similar advantages: share ability of code and data, code and data integrity, centralized management, security, fault tolerance, load balancing, performance, and transaction recovery services

However, CICS has been providing transaction services since 1965 (not the 1990s), and provides scalability, security, and performance levels that that modern web application servers cannot come close. There are twice as many CICS transactions processed in a day that the number of web pages served. The foundation of CICS's dominance is CICS is optimized to take advantage of zEnterprise servers and z/OS. But, of course CICS is not either free or inexpensive.

Traditionally, CICS interfaces with users via a CICS map. A CICS map is a text-based user interface similar in functionally to a HTML web page. CICS maps can be displayed on dumb terminals or terminal emulators. CICS maps requires little processing on the client side. On the other hand, a HTML web page is a complex, multimedia user interface with requires significant processing at the client side. Between 1960 and 1980 there were very few microcomputers. CICS was the only alternative.

The second component of CICS is the appliance programming interface (API). COBOL was originally designed for batch processing. CICS and application services sits between the host operating system and COBOL executable module. In a CICS infrastructure, z/OS executes one or more copies (regions) of CICS. CICS in turn executes one or more COBOL applications. In program terminology, CICS calls a COBOL load module, which in turn the COBOL load module calls (requests) services from other CICS programs. COBOL does not change. But, the application programmer adds statements to the COBOL program contains identified a CICS API block, e.g., EXEC CICS …. CICS API Statements … END-EXEC. Before the COBOL program is compiled these CICS API block are placed by traditional COBOL statements.

Interfacing with CICS is a very complicated coding process. To make the life of a COBOL programmer easier, CICS provides these easy-to-code CICS API blocks. CICS provides APIs for PL1 and Java. All Java web application servers are based on Java, which also includes to the Java Enterprise APIs. While J2EE APIs provides similar functionality to COBOL CICS APIs, both COBOL and CICS APIs are much easier to learn and code.

Advantages of CICS (and many other OLTP systems)

· Immediate execution of a transaction. Users can enter transactions and the system may be updated immediately. Some CICS transactions just stores data in a batch transaction file to be processed later, e.g., a commercial bank in which employees enters checks enter to file and the transactions to update the balances will be processed later. Online processing is difference than real time processing.

· Provides services not provided by the host operating system, e.g., share ability of code and data, code and data integrity, centralized management, security, fault tolerance, load balancing, performance, and transaction recovery services.

· CICS provides APIs for several programming languages, J2EE does not.

· While originally designed to support 3270 terminals over a SNA network, CICS now supports TCP/IP, web pages, mobile applications, and web services

Disadvantages of CICS (and many other OLTP systems)

· Cannot to be scheduled and generally requires resources 24/7. Idle time

· Extra services require extra processing. The cost per online transactions is substantially more expensive than the cost per batch transaction.

· Increase application programming complexity, e.g., user interfaces, user validations, help screens, navigation, etc.

· Complicate configurations are required to balance peak loads and system utilization objectives

IBM WebSphere

IBM WebSphere is IBM's proprietary version of a J2EE server, which provides a web application user interface, i.e., a webpage. Unlike other J2EE web application servers, WebSphere can be executed WebSphere can take advantage of zEnterprise operating systems (z/OS Unix and zLinux), hardware and middleware systems. For example, assume that an organization supports COBOL batch applications, VSAM data sets, DB2 databases, and traditional CICS Maps. WebSphere can permit a web application share the same VSAM dataset or DB2 database as a traditional COBOL batch application. This provides an infrastructure design that supports the existing investment while providing the advantages of extending J2EE functionality.

Advantages of IBM WebSphere

· Provides a J2EE web interface and application server that supports and integrates with the existing investment in zEnterprise.

· Takes advantage in the pool and experience of J2EE programmers

· Provides scalability, security and performance not available to other proprietary J2EE solutions.

· Can interface with CICS

Disadvantages of IBM WebSphere

· Based on the Java program language as used in other J2EE platforms, IBM WebSphere Java applications execute 10-40% slower than COBOL CICS load modules.

· Based on the J2EE application server as used in J2EE platforms, a IBM WebSphere Application server executes 10-20% slower than a CICS transaction monitor.

· Any J2EE platform, including WebSphere, cannot scale or perform as efficiently as CICS

IBM WebSphere MQ

Application programs have historically been able to call (or invoke) another separate application load module and send/receive data. However, these applications must be supported by a common programming language, operating system and methods of communications between distributed systems. While there were several proprietary of program language solutions, today's modern application distributed systems support a collection of web services or REST-full protocols, e.g., XML, SOAP, WSDL, HTTP, JSON

Both WebSphere and CICS were traditionally designed to support remote user-to-application solutions. WebSphere MQ will add a second specialized application server that will support messages sent between applications (not users). Applications based on different programming languages and operating systems can easily pass real-time messages.

Remote distributed applications are not directly linked, rather the share data must be process by a web service application server on both the client and server side. The calling application sends the message to client web service server which stores the message in a queue (memory or data set storage areae). The sending sever may transmit the message immediately (synchronous) or later (asynchronous). Likewise, the server side can process the message immediately or later.

MQ stands for Message Queing. Messages represent the data shared between remote applications. Queues represent the waiting area, e.g., memory or data sets, used to control transaction flow, sequencing and timing of execution, fault tolerance and transaction recovery. Other web service solution includes Apache Axis, WSO2, JBOSS, Oracle Application Server, etc.,

Web applications require a user interface to start a transaction. MQ and web services are started by another application. Web Services is the foundation of supply-chain and business partner ERP services. Companies like BNY-Mellon expect that 50-70% of new application development will use web services.

Introduction to File Systems

File systems - A file system is a collection of system programs that control the manner in which files are named, are created, and data is stored and later retrieved. Each operating system supports one or more file systems.

· Consider the Windows file system named NTFSv5. To name a Window's file one may use up to 255 characters, include embedded spaces, and will retain the case of letters in a file name. The difference between upper and lower-case letters in NTFSv5 does not make the file name unique, e.g., PACKY.DOCX is the same file name as packy.docx.

· Files and z/OS data sets are containers that may store records. Records contain structured information such as the employee name, address, and phone number of an employee information file. Files and z/OS records may also contain unstructured information as a collection bytes; such as graphic, audio and video files.

· File systems specify rules for creating a file or data set. In Windows NTFSv5, a file is automatically created when you select SAVE for the first time. When created a file must be named, a file type must be assigned, a stored location is selected, storage space is allocated to the container, and the data content is then stored in the file container, e.g., the content of a Word document is stored in a file of the type named .DOCX.

· The file system does not understand the details of the content of a file or data set. Remember a file is a container. The applications, e.g., Word, a COBOL Program, etc. that store the data in the file container will understand the structure of the content, not the operating system or file system.

· Every file system has structure that organizes files and improves the efficiency to find and locate files. Most modern hard disk or solid state drives (DASD) may individual store billions of files and the files system may organize files across thousands of distributed hard drives. I can't remember which drawer I put (stored) my socks. Can you imagine the complexity of organizing files in today's file systems.

· Windows, UNIX and Linux organize files by storing them in a file folder or directory. A file folder or directory is similar to the old-fashion card catalog of a library. A library catalog contains the name of book and provides information concerning the location of the book in the library or located at other libraries. Today we simple use Google to find a location of the restaurant. Windows, UNIX, and Linux use a hierarchical file system. Within a hierarchical file system a file folder may contain a subfolder, and a subfolder will contain lower-level subfolder.

· IBM's z/FS is a flat file system, and it is not hierarchical. The closet thing that z/OS has to a folder is a Partition Data Set (PDS) - to be discussed next. While a z/FS PDS organizes and contains members, a PDS dataset can only contain sequential members, but cannot contain another PDS. Hierarchical files systems can organize files at multiple levels, but a flat file system can at best organize files (data sets) at one level.

· While a hierarchical file system can better organize a large number of files better, the performance to locate a file is slower than the z/FS. When z/OS needs to organize a large number of files one can use the zEnterprise UNIX file systems. The z/FS and UNIX files systems can be interchangeable depending on the storage objectives, e.g., performance versus organization.

· File systems frequently contain file security data stored in folders/subdirectories or the properties of file. The z/FS does not directly contain security data. RACF, the z/OS security server, will store data set security data based on the data set name, e.g., the HLQ.

Data Sets and Data Set Organization

Data sets - In z/OS, a data set is a named collection of related data records that are retrieved and stored by an assigned name. A data set is equivalent to a file as used in the Windows, UNIX and Linux file systems.

A data set is a container. The manner in which z/OS structures the internal content of the data set is called the data set organization . JCL uses the attribute DSORG to specify the data set organization. There are two popular types of z/OS data set organization: Sequential Data Sets (PS organization) and Partitioned Data Set (PO organization).

Sequential Data Sets (PS organization) - A sequential data set is a collection of records written and read in sequential order from beginning to end.

In z/OS, a sequential data set will normally contain data to be processed by a program, e.g., an customer data set. Each record will contain one customer. The above information is displayed for sequential data set named RMUI001 DATA.ACCOUNTS. Each student has a copy of sequential data set. Each customer record is 170 characters long. The first 72 characters of each customer record is displayed below.

Partitioned Data Set (PO organization) - A data set that contains a directory and sequential members.

When you create a sequential dataset (PS), it occupies a large amount of storage space (minimum 50,000 bytes). Sometimes for doing trivial tasks, storing only a few records, you would not be utilizing the entire space available in a sequential file. Thus most of the space in the Sequential file is wasted and is blocked.

IBM provides a way to split-up or partition the space in a sequential data-set. IBM Software engineers invented the Partitioned Dataset, often called PDS. Each part is called a member. A Partitioned Dataset can have many members (parts). Each member behaves like a sequential file on its own.

How does the PDS then keep track of its members? A Partitioned Dataset (PDS) includes a directory Similar to Windows and Linux. The members of a PDS can be scattered hap hazardly anywhere in the vast computer storage space. Well-then, you consult the directory to find out the members in the PDS. Just as in a book, the index tells you, the keyword you are looking for and the page no. where it can be found, the same way, the directory tells you the member name and the computer memory address where you can find it. Thus, the directory maintains the list of member-names and pointers to the actual physical place in computer memory, where they are stored.

Not just this, but the directory also keeps track of its statistics, when the member was created, when was it last modified/viewed by somebody, how much space it occupies, the TSO user-id that created it amongst other things

On the Windows Operating System, we generally keep related files together in a folder. On the same lines, you keep related members together in one PDS. What a folder is to Windows OS, a Partitioned dataset (PDS) is to z/OS. Folders contain many related files. A PDS contains many related members.