Posts Tagged ‘hadoop’

Hadoop Brings Big Data Analytics to the IBM System z

October 16, 2014

In a previous blog, DancingDinoaur reported on IBM’s initial announcement of Hadoop and other analytic products, like InfoSphere BigInsights, coming to the z. The IBM announcement itself can be found here.

Subsequent sessions at IBM Enterprise2014 delved more deeply into big data, analytics, and real-time analytics. A particularly good series of sessions was offered by Karen Durward, an IBM InfoSphere software product manager specializing in System z data integration. As Durward noted, BigInsights is Apache Hadoop wrapped up to make it easier to use for general IT and business managers.

Specifically, the real-time analytics package for z includes IBM InfoSphere BigInsights for Linux on System z, which combines open-source Apache Hadoop with enhancements to make Hadoop System z enterprise-ready. The solution also includes IBM DB2 Analytics Accelerator (IDAA), which improves data security while delivering a 2000x faster response time for complex data queries.

In her Hadoop on z session, Durward started with the Hadoop framework, which consists of four components:

  1. Common Core—the basic modules (libraries and utilities) on which all components are built
  2. Hadoop Distributed File System (HDFS)—stores data on multiple machines to provide very high aggregate bandwidth across a cluster of machines
  3. MapReduce—the programming model to support the high data volume data processing by the cluster
  4. YARN (Yet Another Resource Negotiator)—the platform used to manage the cluster’s compute resources including scheduling users’ applications. In effect, YARN decouples Hadoop workload and resource management.

The typical Hadoop process sounds deceptively straightforward.  Simply load data into an HDFS cluster, analyze the data in the cluster using MapReduce, write the resulting analysis back into the HDFS cluster. Then just read it.

Sounds easy enough until you try it. Then you need to deal with client nodes and name nodes, exchange metadata, and more. In addition, Hadoop is an evolving technology. Apache continues to add pieces to the environment in an effort to simplify it. For instance, Hive provides the Apache data warehouse framework, accessible using HivQL, and HBase brings Apache’s Hadoop database. Writing Map/Reduce code is a challenge so there is Pig, Apache’s platform for creating long and deep Hadoop source programs, and the list goes on. In short, Hadoop is not easy, especially for IT groups accustomed to relational databases and SQL. That’s why you need tools like BigInsights. The table below is how Durward sees the Hadoop tool landscape.

Software Needs Other Hadoop Products BigInsights
Open Source Apache Hadoop Y Y
Rich SQL on Hadoop (Big SQL) some Y
Tools for Business Users (BigSheets) NA Y
Advanced text analytics NA Y
In-Hadoop analytics NA Y
Rich developer tools NA Y
Enterprise workload & storage mgt. NA Y
Comprehensive suite NA Y

In fact, you need more than BigInsights. “We don’t know how to look at unstructured data,” said Durward. That’s why IBM layers on tools like Big SQL, which helps you query Hadoop’s HBase using industry-standard SQL. You can migrate a relational table to HBase using Big SQL or connect Big SQL via JDBC to run business intelligence and reporting tools, such as Cognos, which also runs on Linux on z. Similarly IBM offers BigSheets, a cloud application that performs ad hoc analytics at web-scale on unstructured and structured content using the familiar spreadsheet format.

Lastly, Hadoop queries often produce free-form text, which requires text analytics to make sense of the results. Not surprisingly, IBM offers BigInsights Text Analytics, a fast, declarative rule-based information extraction (IE) system that extracts insights from unstructured content. This system consists of a fast, efficient runtime that exploits numerous optimization techniques across extraction programs written in Annotation Query Language (AQL), an English-like declarative language for rule-based information extraction.

Hadoop for the z is more flexible than z data center managers may think. You can merge Hadoop data with z transactional data sources and analyze it all together through BigInsights.

So how big will big data be on the z? DancingDinosaur thought it could scale to hundreds of terabytes, even petabytes. Not so. You should limit Hadoop on the z to moderate volumes—from hundreds of gigabytes to tens of terabytes, Durward advises, adding “after that it gets expensive.”

Still, there are many advantages to running Hadoop on the z. To begin, the z brings rock solid security, is fast to deploy, and, through BigInsights, brings an easy-to-use data ingestion process. It also has proven to be easy to setup and run, taking just a few hours, with conversions handled automatically. Lastly, the data never leaves the platform, which avoids the expense and delay of moving data between platforms. But maybe most importantly, by wrapping Hadoop in a set of familiar, comfortable tools and burying its awkwardness out of sight Hadoop now becomes something every z shop can leverage.

DancingDinosaur is Alan Radding. Follow this blog on Twitter, @mainframeblog. Check out my work at Technologywriter.com

Software Licensing for IBM System z Distributed Linux Middleware

October 10, 2014

DancingDinosaur can’t attend a mainframe conference without checking out at least one session on mainframe software pricing by David Chase, IBM’s mainframe pricing guru. At IBM Enterprise2014, which wraps up today, the topic of choice was software licensing for Linux middleware. It’s sufficiently complicated to merit an entire session.

In case you think Linux on z is not in your future, maybe you should think again.  Linux is gaining momentum in even the largest z data centers. Start with IBM bringing new apps like InfoSphere, BigInsights (Hadoop), and OpenStack to z. Then there are apps from ISVs that just weren’t going to get their offerings to z/OS. Together it points to a telltale sign something is happening with Linux on z. And, the queasiness managers used to have about the open source nature of Linux has long been put to rest.

At some point, you will need to think about IBM’s software pricing for Linux middleware. Should you find yourself getting too lost in the topic, check out these links recommended by Chase:

To begin, software for Linux on z is treated differently than traditional mainframe software in terms of pricing. With Linux on z you think in terms of IFLs.  The quantity of IFLs represent the number of Linux engines subjected to IBM’s IPLA-based pricing.

Also think in terms of Processor Value Units (PVUs) rather than MSUs. For a pricing purposes, PVUs are analogous to MSUs although the values are different. A key point to keep in mind: distributed PVUs for Linux are not related to System z IPLA value units used for z/VM products. As is typical of IBM, those two different kinds of value units are NOT interchangeable.

Chase, however, provides a few ground rules:

  • Dedicated partition
    • Processors are always allocated in whole increments
    • Resources are only moved between partitions “explicitly” (e.g. by an operator or a scheduled job)
  • Shared pool:
    • Pool of processors shared by partitions (including virtual machines)
    • System automatically dispatches processor resources between partitions as needed
  • Maximum license requirements
  • Customer does not have to purchase more licenses for a product than the number of processors on the machine (e.g. maximum DB2 UDB licenses on a 12-way machine is 12)
    • Customer does not have to purchase more “shared pool” licenses for a product than the number of processors assigned to the shared pool (e.g. maximum of 7 MQSeries licenses for a shared pool with 7 processors). Note: This limit does not affect the additional licenses that might be required for dedicated partitions.

With that, as Chase explains it, Linux middleware pricing turns out to be relatively straightforward, determined by:

  • Processor Value Unit (PVU) rating for each kind of core
  • Any difference for different processor technologies (p, i, x, z, Sun, HP, AMD, etc—notice that the z is just one of many choices, not handled differently from the others
  • Number of processor cores which must be licensed (z calls them IFLs)
  • Price per PVU (constant per product, not different based upon technology)

Then it becomes a case of doing the basic arithmetic. The formula: # of PVUs x the # of cores required x the value ($) per core = your total cost.  Given this formula it is to your advantage to plan your Linux use to minimize IFLs and cores. You can’t do anything about the cost per PVU.

Distributed PVUs are the basis for licensing middleware on IFLs and are determined by the type of machine processor. The zEC12, z196, and z10 are rated at 120 PVUs. All others are rated at 100 PVUs. For example, any distributed middleware running on Linux on z this works out to:

  • z114—1IFL, 100 PVUs
  • z196—4IFLs, 480 PVUs
  • zEC12—8 IFLs, 960 PVUs

Also, distributed systems Linux middleware offerings are eligible for sub-capacity licensing. Specifically, sub-capacity licensing is available for all PVU-priced software offerings that run on:

  • UNIX (AIX, HP-UX, and Sun Solaris
  • i5/OS, OS/400
  • Linux (System i, System p, System z)
  • x86 (VMware ESX Server, VMware GSX Server, Microsoft Virtual Server)

IBM’s virtualization technologies also are included in Passport Advantage sub-capacity licensing offering, including LPAR, z/VM virtual machines in an LPAR, CPU Pooling support introduced in z/VM 6.3 APAR VM65418, and native z/VM (on machines which still support basic mode).

And in true z style, since this can seem more complicated than it should seem, there are tools available to do the job. In fact Chase doesn’t advise doing this without a tool. The current tool is the IBM License Metric Tool V9.0.1. You can find more details on it here.

If you are considering distributed Linux middleware software or are already wrestling with the pricing process, DancingDinosaur recommends you check out Chase’s links at the top of this piece. Good luck.

DancingDinosaur is Alan Radding. Follow DancingDinosaur on Twitter, @mainframeblog. You can check out more of my work at Technologywriter.com

Real-Time Analytics on z Lead at IBM Enterprise2014 Opening Day

October 8, 2014

Users have always been demanding about performance. But does the 5-minute rule noted by Tom Rosamilia in the opening keynote at IBM Enterprise2014 go too far? It now seems users expect companies to respond, or at least acknowledge, their comments, questions, or problems in five minutes. That means companies need to monitor and analyze social media in real-time and respond appropriately.

Building on client demand to integrate real-time analytics with consumer transactions, IBM yesterday announced new capabilities for its System z. Specifically, IBM is combining the transactional virtues of the z with big data analytic capabilities into a single, streamlined, end-to-end data system. This real-time integration of analytics and transaction processing can allow businesses to increase the value of a customer information profile with every interaction the customer makes.  It also promises one way to meet the 5-minute rule, especially when a customer posts a negative comment on social media.

With the new integrated capability you can apply analytics to social sentiment and customer engagement data almost as the transactions are occurring. The goal is to gain real-time insights, which you can do on the mainframe because the data already is there and now the real time analytics will be there. There is no moving of data or logic.  The mainframe already is doing this when it is being used for fraud prevention. This becomes another case where the mainframe can enable organizations to achieve real-time insights and respond within five minutes. Compared to fraud analysis the 5-minute expectation seems a luxury.

By incorporating social media into the real time analytic analysis on the mainframe you can gain an indication of how the business is performing in the moment, how you stack up to your competitors, and most importantly, meet the 5-minute response expectation.  Since we’re talking about pretty public social sentiment data, you also could monitor your competitors’ social sentiment and analyze that to see how well they are responding.

And then there are the more traditional things you can do with the integration of analytics with transactional data to provide real-time, actionable insights on commercial transactions as they occur. For example you could take advantage of new opportunities to increase sales or prevent customer churn.

According to IBM this is being driven by the rise of mobile and smartphones, numbering in the billions in a few years. The combination of massive amounts of data and consumers who are empowered with mobile access is creating a difficult challenge for businesses, IBM noted in the announcement. Consumers now expect an immediate response—the 5 minute rule—to any interaction, at any time, and through their own preferred channel of communication. Unfortunately, many businesses are trying to meet this challenge and deliver instantaneous, on-demand customer service with outdated IT systems that can only provide after-the-fact intelligence.

Said Ross Mauri, General Manager, System z, IBM Systems & Technology Group: “Off-loading operational data in order to perform analytics increases cost and complexity while limiting the ability of businesses to use the insights in a timely manner.” The better approach, he continued, is to turn to an end-to-end solution that makes analytics a part of the flow of transactions and allows companies to gain real time insights while improving their business performance with every transaction.

Of course,  Mauri was referring specifically to the System z.  However, Power Systems and especially the new POWER8 machines, which have a strong presence here at IBM Enterprise2014, can do it too. Speaker after speaker emphasized that the Power machines are optimized for lightning fast analytics, particularly real time analytics.

Still, this was a z announcement so IBM piled on a few more goodies for the z. These include new analytics capabilities for the mainframe to enable better data security and provide companies with the ability to integrate Hadoop big data with the z. Specifically, IBM is delivering:

  • IBM InfoSphere BigInsights for Linux on System z – Combines open-source Apache Hadoop with IBM innovations to deliver enterprise grade Hadoop for System z clients;
  • IBM DB2 Analytics Accelerator – Enhances data security while delivering 2000 times the response time for complex data queries.
  • New capabilities in Linux and the cloud for system z, such as IBM Elastic Storage for Linux on System z, which extends the benefits of Elastic Storage to the Linux environment on z servers, and IBM Cloud Manager with OpenStack for System z, which enables heterogeneous cloud management across System z, Power and x86 environments.

Many of these pieces are available now.  You can meet the 5-minute rule sooner than you may think.

Alan Radding is DancingDinosaur. Follow him on Twitter, @mainframeblog, or check out his website, Technologywriter.com

 

IBM Enterprise2014 to Drive Advanced Mainframe Capabilities

August 27, 2014

The summer is winding down, and IBM Enterprise2014 (October 6-10, 2014 at the Venetian in Las Vegas, October 6-10, 2014 at the Venetian in Las Vegas) will be here in a little over a month.  It combines the IBM System z Technical University and the IBM Power Systems Technical University at one North American location. The advanced capabilities being featured at Enterprise2014 include: cloud, big data, and much more. Let’s look at a sampling of the z-oriented cloud and big data sessions. Subsequent posts will look at POWER and other topics.

The event also will include announcing the winner of the Mainframe Mobile App Throwdown, details here. Mobile is hot and poised to drive a lot of activity through the mainframe. The next generation of mobile apps will need to integrate with core applications running on the mainframe. DancingDinosaur readers know how to do that. Top prize for the Throwdown is an iPad, a pass to the IBM Enterprise2014 conference in Las Vegas, and even a week with IBM experts to help turn the app from a concept to reality. DancingDinosaur will be there to publicize the winners here. But the competition closes Sept. 17 so sign up soon.

For Mainframe Mobile App Throwdown ideas check out the session details at Enterprise2014. For example, Taking Analytics Mobile with DB2 Web Query and More! by Doug Mack digs into mobile features added to DB2 Web Query. He discusses how to sync a mobile device up with your favorite dashboards, or use the mobile app to organize and access reports offline. Leverage REST-based Web Services and application extensions to customize the user interface for reporting functions or schedule the reports to run in the background.

Now, let’s look at a sampling of the cloud and big data sessions.

How Companies Are Using IBM System z for Cloud—Fehmina Merchant describes how organizations are building secure and robust private clouds on System z to deliver their critical IT services with agility and at lower costs.  The session will examine the unique capabilities of zEnterprise as a platform for private cloud computing, in effect providing the ultimate in virtualization, security, scalability and reliability. It also will cover how the newest IBM SmartCloud technologies can automate and optimize the deployment and management of services in the cloud. In addition, the session will offer some specific real-life examples and use-cases to illustrate how a private cloud built on zEnterprise and SmartCloud provide flexible IT service delivery at the lowest cost. The session will end with a live demonstrations of the latest IBM SmartCloud tools.

Should mainframe shops even care about cloud computing? That’s a question DancingDinosaur gets asked frequently. Glenn Anderson answers it in zEnterprise—Cutting Through the Hype: Straight Talk About the Mainframe and Cloud Computing. In this session he promises to explain why the cloud is relevant to a System z enterprise and helps z data center managers cut through the marketing hype.

For zLinux there is The Elephant on the Mainframe—Using Hadoop to Analyze IBM System z Data by Christopher Spaight. He describes the zEnterprise portfolio as including a rich set of options for the analysis of structured, relational data. But what, he asks, if the business needs to analyze data that is unstructured or semi-structured or a mix of relational and non-relational records? Many are looking to Hadoop in these situations. This session lays out the mainframer’s options for using Hadoop both on and off platform, and walks through several use cases for when it makes sense to use Hadoop. BTW, Hadoop on z is called zDoop.

Finally, HDFS, Hive and All That Big Data “Stuff” for IBM System z by Karen Durward looks at how the System z participates in the world of HDFS, Hive and more Big Data stuff. This session focuses on not only why z/OS data should be integrated into a Big Data environment but the various ways to do it. She will describe the latest on z/OS data integration with Big Data, Linux on System z as a Big Data platform, and more.

Then, when you have absorbed all the technology you can enjoy three evenings of live performances: 2 country rock groups, Delta Rae and The Wild Feathers and then, Rock of Ages. Check ‘em out here.

Alan Radding is DancingDinosaur. Look for me at Enterprise2014. You can follow this blog and more on Twitter, @mainframeblog. Find Alan Radding on Technologywriter.com.

Industrial Strength SDS for the Cloud

June 12, 2014

The hottest thing in storage today is software defined storage (SDS). Every storage vendor is jumping on the SDS bandwagon.

The presentation titled Industrial -Strength SDS for the Cloud, by Sven Oehme, IBM Senior Research Scientist, drew a packed audience at Edge 2014 and touched on many of the sexiest acronyms in IBM’s storage portfolio.  These included not just GPFS but also GSS (also called GPFS Storage Server), GNR, LROC (local read-only cache), and even worked in Linear Tape File System (LTFS).

The session promised to outline the customer problems SDS solves and show how to deploy it in large scale OpenStack environments with IBM GPFS.  Industrial strength generally refers to large-scale, highly secure and available multi-platform environments.

The session abstract explained that the session would show how GPFS enables resilient, robust, reliable, storage deployed on low-cost industry standard hardware delivering limitless scalability, high performance, and automatic policy-based storage tiering from flash to disk to tape, further lowering costs. It also promised to provide examples of how GPFS provides a single, unified, scale-out data plane for cloud developers across multiple data centers worldwide. GPFS unifies OpenStack VM images, block devices, objects, and files with support for Nova, Cinder, Swift and Glance (OpenStack components), along with POSIX interfaces for integrating legacy applications. C’mon, if you have even a bit of IT geekiness, doesn’t that sound tantalizing?

One disclaimer before jumping into some of the details; despite having written white papers on SDS and cloud your blogger can only hope to approximate the rich context provided at the session.

Let’s start with the simple stuff; the expectations and requirements for cloud  storage:

  • Elasticity, within and across sites
  • Secure isolation between tenants
  • Non-disruptive operations
  • No degradation by failing parts as components fail at scale
  • Different tiers for different workloads
  • Converged platform to handle boot volumes as well as file/object workload
  • Locality awareness and acceleration for exceptional performance
  • Multiple forms of data protection

Of course, affordable hardware and maintenance is expected as is quota/usage and workload accounting.

Things start getting serious with IBM’s General Parallel File System (GPFS). This what IBMers really mean when they refer to Elastic Storage, a single name space provided across individual storage resources, platforms, and operating systems. Add in different classes of storage devices (fast or slow disk, SSD, Flash, even LTFS tape), storage pools, and policies to control data placement and you’ve got the ability to do storage tiering.  You can even geographically distribute the data through IBM’s Active Cloud Engine, initially a SONAS capability sometimes referred to as Active File Manager. Now you have a situation where users can access data by the same name regardless of where it is located. And since the system keeps distributed copies of the latest data it can handle a temporary loss of connectivity between sites.

To protect the data add in declustered software RAID, aka GNR or even GSS (GPFS Storage Server). The beauty of this is it reduces the space overhead of replication through declustered parity (80% vs. 33% utilization) while delivering extremely fast rebuild.  In the process you can remove hardware storage controllers from the picture by doing the migration and RAID management in software on your commodity servers.

dino industrial sds 1

In the above graphic, focus on everything below the elongated blue triangle. Since it is being done in software, you can add an Object API for object storage. Throw in encryption software. Want Hadoop? Add that too. The power of SDS.  Sweet

The architecture Oehme lays out utilizes generic servers with direct-attached switched JBOD (SBOD). It also makes ample use of LROC, which provides a large read cache that benefits many workloads, including SPECsfs, VMware, OpenStack, other virtualization, and database workloads.

A key element in Oehme’s SDS for the cloud is OpenStack. From a storage standpoint OpenStack Cinder, which provides access to block storage as if it were local, enables the efficient sharing of data between services. Cinder supports advanced features, such as snapshots, cloning, and backup. On the back end, Cinder supports Linux servers with iSCSI and LVM; storage controllers; shared filesystems like GPFS, NFS, GlusterFS; and more.

Since Oehme’s  is to produceindustrial-strength SDS for the Cloud it needs to protect data. Data protection is delivered through backups, snapshots, cloning, replication, file level encryption, and declustered RAID, which spans all disks in the declustered array and results in faster RAID rebuild (because there are more disks available for RAID rebuild.)

The result is highly virtualized, industrial strength SDS for deployment in the cloud. Can you bear one more small image that promises to put this all together? Will try to leave it as big as can fit. Notice it includes a lot of OpenStack components connecting storage elements. Here it is.

dino industrial sds 2

DancingDinosaur is Alan Radding. Follow DancingDinosaur on Twitter @mainframeblog

Learn more about Alan Radding at technologywriter.com

Happy 50th System z

April 11, 2014

IBM threw a delightful anniversary party for the mainframe in NYC last Tuesday, April 8. You can watch video from the event here

About 500 people showed up to meet the next generation of mainframers, the top winners of the global Master of the Mainframe competition. First place went to Yong-Sian Shih, Taiwan; followed by Rijnard van Tonder, South Africa; and Philipp Egli, United Kingdom.  Wouldn’t be surprised if these and the other finalists at the event didn’t have job offers before they walked out of the room.

The System z may be built on 50-year old technology but IBM is rapidly driving the mainframe forward into the future. It had a slew of new announcements ready to go at the anniversary event itself and more will be rolling out in the coming months. Check out all the doings around the Mainframe50 anniversary here.

IBM started the new announcements almost immediately with Hadoop on the System z. Called  zDoop, the industry’s first commercial Hadoop for Linux on System z, puts map reduce big data analytics directly on the z. It also announced Flash for mainframe, consisting of the latest generation of flash storage on the IBM DS8870, which promises to speed time to insight with up to 30X the performance over HDD. Put the two together and the System z should become a potent big data analytics workhorse.

But there was even more. Mobile is hot and the mainframe is ready to play in the mobile arena too. Here the problem z shops experience is cost containment. Mainframe shops are seeing a concurrent rise in their costs related to integrating new mobile applications. The problem revolves around the fact that many mobile activities use mainframe resources but don’t generate immediate income.

The IBM System z Solution for Mobile Computing addresses this with new pricing for mobile workloads on z/OS by reducing the cost of the growth of mobile transaction volumes that can cause a spike in software charges. This new pricing will provide up to a 60% reduction on the processor capacity reported for Mobile activity, which can help normalize the rate of transaction growth that generates software charges. The upshot: much mobile traffic volume won’t increase your software overhead.

And IBM kept rolling out the new announcements:

  • Continuous Integration for System z – Compresses the application delivery cycle from months to weeks or days.   Beyond this IBM suggested upcoming initiatives to deliver full DevOps capabilities for the z
  • New version of IBM CICS Transaction Server – Delivers enhanced mobile and cloud support for CICS, able to handle more than 1 billion transactions per day
  • IBM WebSphere Liberty z/OS Connect—Rapid and secure enablement of web, cloud, and mobile access to z/OS assets
  • IBM Security zSecure SSE – Helps prevent malicious computer attacks with enhanced security intelligence and compliance reporting that delivers security events to QRadar SIEM for integrated enterprise- wide security intelligence dashboarding

Jeff Frey, an IBM Fellow and the former CTO of System z, observed that “this architecture was invented 50 years ago, but it is not an old platform.”  It has evolved over those decades and continues evolve. For example, Frey expects the z to accommodate 22nm chips and a significant increase in the increase in the number of cores per chip. He also expects vector technology, double precision floating point and integer capabilities, and FPGA to be built in. In addition, he expects the z to include next generation virtualization technology for the cloud to support software defined environments.

“This is a modern platform,” Frey emphasized. Other IBMers hinted at even more to come, including ongoing research to move beyond silicon to maintain the steady price/performance gains the computing industry has enjoyed the past number of decades.

Finally, IBM took the anniversary event to introduce a number of what IBM calls first-in-the-enterprise z customers. (DancingDinosaur thinks of them as mainframe virgins).  One is Steel ORCA, a managed service provider putting together what it calls the first full service digital utility center.  Based in Princeton, NJ, Phase 1 will offer connections of less than a millisecond to/from New York and Philadelphia. The base design is 300 watts per square foot and can handle ultra-high density configurations. Behind the operation is a zEC12. Originally the company planned to use an x86 system but the costs were too high. “We could cut those costs in half with the z,” said Dave Crocker, Steel ORCA chairman.

Although the Mainframe50 anniversary event has passed, there will be Mainframe50 events and announcements throughout the rest of the year.  Again, you can follow the action here.

Coming up next for DancingDinosaur is Edge2014, a big infrastructure innovation conference. Next week DancingDinosaur will look at a few more of the most interesting sessions, and there are plenty. There still is time to register. Please come—you’ll find DancingDinosaur in the bloggers lounge, at program sessions, and at the Sheryl Crow concert.

Follow DancingDinosaur on Twitter, @mainframeblog

 

Enterprise 2013 Details System z and Power Technology and New Capabilities

October 25, 2013

IBM announced a lot of goodies for z and Power users at Enterprise 2013 wrapping up in Orlando today. There were no blockbuster announcements, like a new z machine—we’re probably 12-18 months away from that and even then the first will likely focus on Power8—but it brought a slew of announcements nonetheless. For a full rundown on what was announced click here.

Cloud and analytics—not surprisingly—loom large. For example, Hadoop and a variety of other capabilities have been newly cobbled together, integrated, optimized, and presented as new big data offerings or as new cloud solutions.  This was exemplified by a new Cognos offering for CFOs needing to create, analyze and manage sophisticated financial plans that can provide greater visibility into enterprise profitability or the lack thereof.

Another announcement featured a new IBM Entry Cloud Configuration for SAP on zEnterprise. This is a cloud-enablement offering combining high-performance technology and services to automate, standardize and accelerate day-to-day SAP operations for reduced operational costs and increased ROI. Services also were big at the conference.

Kicking off the event was a dive into data center economics by Steve Mills, Senior Vice President & Group Executive, IBM Software & Systems. Part of the challenge of optimizing IT economics, he noted, was that the IT environment is cumulative. Enterprises keep picking up more systems, hardware and software, as new needs arise but nothing goes away or gets rationalized in any meaningful way.

Between 2000 and 2010, Mills noted, servers had grown at a 6x rate while storage grew at a 69x rate. Virtual machines, meanwhile, were multiplying at the rate of 42% per year. Does anyone see a potential problem here?

Mills’ suggestion: virtualize and consolidate. Specifically, large servers are better for consolidation. His argument goes like this: Most workloads experience variance in demand. But when you consolidate workloads with variance on a virtualized server the variance of the sum is less due to statistical multiplexing (which fits workloads into the gaps created by the variances). Furthermore, the more workloads you can consolidate, the smaller the variance of the sum. His conclusion: bigger servers with capacity to run more workloads can be driven to higher average utilization levels without violating service level agreements, thereby reducing the cost per workload. Finally, the larger the shared processor pool is the more statistical benefit you get.

On the basis of statistical multiplexing, the zEnterprise and the Power 795 are ideal choices for this. Depending on your workloads, just load up the host server, a System z or a big Power box, with as many cores as you can afford and consolidate as many workloads as practical.

Mills’ other cost savings tips: use flash to avoid the cost and complexity of disk storage. Also, eliminate duplicate applications—the fewer you run, the lower the cost. In short, elimination is the clearest path to saving money in the data center.

To illustrate the point, Jim Tussing from Nationwide described how the company virtualized and consolidated 60% on their 10,500 servers on a few mainframes and saved $46 million over five years. It also allowed the company to delay the need for an additional data center for 4 years.

See, if DancingDinosaur was an actual data center manager it could have justified attendance at the entire conference based on the economic tips from just one of the opening keynotes and spent the rest of the conference playing golf. Of course, DancingDinosaur doesn’t play golf so it sat in numerous program sessions instead, which you will hear more about in coming weeks.

You can follow DancingDinosaur on twitter, @mainframeblog

Big Data as a Game Changing Technology at IBM Edge 2013

June 11, 2013

If you ever doubted that big data was going to become important, there should be no doubt anymore. Recent headlines from the past couple of weeks of the government capturing and analyzing massive amounts of daily phone call data should convince you. That this report was shortly followed by more reports of the government tapping the big online data websites like Google, Yahoo, and such for even more data should alert you to three things:

1—There is a massive amount of data out there that can be collected and analyzed.

2—Companies are amassing incredible volumes of data in the normal course of serving people who readily and knowingly give their data to these organizations. (This blogger is one of those tens of million .)

3—The tools and capabilities are mature enough for someone to sort through that data and connect the dots to deliver meaningful insights.

Particularly with regard to the last point this blogger thought the industry was still five years away from generating meaningful results from that amount of data coming in at that velocity. Sure, marketers have been sorting and correlating large amounts of data for years, but it was mostly structured data and not at nearly this much. BTW, your blogger has been writing about big data for some time.

If the news reports weren’t enough it became clear at Edge 2013 that big data analytics is happening and companies like Constant Contact and many others are succeeding at it now. It also is clear that there is sufficient commercial off-the-shelf computing power from companies like IBM and analytics tools to sort through massive amounts of data and make sense of it fast.

Another interesting point came up in one of the many discussions touching on big data. Every person’s personal data footprint is as unique as a fingerprint or other bio-metrics. We all visit different websites and interact with social media and use our credit and debit cards in highly individual ways. Again, marketers have sensed this at some level for years, but they haven’t yet really honed it down to the actual individual on a mass scale, although there is no technical reason one couldn’t.

Subsequent blogs will take up other topics from Edge 2013, such as software defined everything.

Although there were over a dozen sessions on System z topics, the mainframe did not have a big presence at the conference. However, Enterprise Systems 2013 was being promoted at IBM Edge. It will take place Oct. 21-25 in Orlando, Fl. It will combine the System z and the Power System Technical University along with a new executive-focused Enterprise Systems event. It will include new announcements, peeks into trends and directions, over 500 expert technical sessions across 10 tracks, and a comprehensive solution center.

IBM Technical Edge 2013 Tackles Flash – Big Data – Cloud & More

June 3, 2013

IBM Edge 2013 kicks off in just one week, 6/10 and runs through 6/14. Still time to register.  This blogger will be there through 6/13.  You can follow me on Twitter for conference updates @Writer1225.  I’ll be using hashtag #IBMEdge to post live Twitter comments from the conference. As noted here previously I’ll buy a drink for the first two people who come up to me and say they read DancingDinosaur.  How’s that for motivation!

The previous post looked at the Executive track. Now let’s take a glimpse at the technical track, which ranges considerably wider, beyond the System z to IBM’s other platforms, flash, big data, cloud, virtualization, and more

Here’s a sample of the flash sessions:

Assessing the World of Flash looks at the key competitors, chief innovators, followers, and leaders. You’ll quickly find that not all flash solutions are the same and why IBM’s flash strategy stands at the forefront of this new and strategic technology.

There are many ways to deploy flash. This session examines Where to Put Flash in the Data Center.  It will focus particularly on the new IBM FlashSystem products and other technologies from IBM’s Texas Memory Systems acquisition. However, both storage-based and server-based flash technologies will be covered with an eye toward determining what works best for client performance needs.

The session on IBM’s Flash Storage Future will take a look at how IBM is leveraging its Texas Memory Systems acquisition and other IBM technologies to deliver a flash portfolio that will play a major role across not only IBM’s storage products but its overall solution portfolio and its roadmap moving forward.

The flash sessions also will look at how Banco Azteco, Thompson Reuters, and Sprint are deploying and benefiting from flash.

In the big data track, the Future of Analytics Infrastructure looks interesting. Although most organizations understand the value of business analytics many don’t understand how the infrastructure choices they make will impact the success or failure of their analytics projects.  The session will identify the key requirements of any analytical environment: agility, scalability, multipurpose, compliance, cost-effective, and partner-ready; and how they can be met within a single, future-ready analytics infrastructure to meet the needs of current and future analytics strategies.

Big data looms large at the conference. A session titled Hadoop…It’s Not Just about Internal Storage explores how the Hadoop MapReduce approach is evolving from server internal disks to external storage. Initially, Hadoop provided massively scalable, distributed file storage and analytic capabilities. New thinking, however, has emerged that looks at a tiered approach for implementing the Hadoop framework with external storage. Understanding the workload architectural considerations is important as companies begin to integrate analytic workloads to drive higher business value. The session will review the workload considerations to show why an architectural approach makes sense and offer tips and techniques, and share information about IBM’s latest offerings in this space.

An Overview of IBM’s Big Data Strategy details the company’s industrial-strength big data platform to address the full spectrum of big data business opportunities. This session is ideal for those who are just getting started with big data.

And no conference today can skip the cloud. IBM Edge 2013 offers a rich cloud track. For instance, Building the Cloud Enabled Data Center explains how to get maximum value out of an existing virtualized environment through self-service delivery and optimization along with virtualization optimization capabilities. It also describes how to enable business and infrastructure agility with workload optimized clouds that provide orchestration across the entire data center and accelerate application updates to respond faster to stakeholder demands and competitive threats. Finally it looks at how an open and extensible cloud delivery platform can fully automate application deployment and lifecycle management by integrating compute, network, storage, and server automation.

A pair of sessions focus on IBM Cloud Storage Architectures and Understanding IBM’s Cloud Options. The first session looks at several cloud use cases, such as storage and systems management.  The other session looks at IBM SmartCloud Entry, SmartCloud Provisioning, and ServiceDelivery Manager.  The session promises to be an excellent introduction for the cloud technical expert who desires a quick overview of what IBM has to offer in cloud software and the specific value propositions for its various offerings, along with their architectural features and technical requirements.

A particularly interesting session will examine Desktop Cloud through Virtual Desktop Infrastructure and Mobile Computing. The corporate desktop has long been a costly and frustrating challenge complicated even more by mobile access. The combination of the cloud and Virtual Desktop Infrastructure (VDI) provides a way for companies to connect end users to a virtual server environment that can grow as needed while mitigating the issues that have frustrated desktop computing, such as software upgrades and patching.

There is much more in the technical track. All the main IBM platforms are featured, including PureFlex Systems, the IBM BladeCenter, IBM’s Enterprise X-Architecture, the IBM XIV storage system, and, for DancingDinosaur readers, sessions on the DS8000.

Have you registered for IBM Edge 2013 yet?  There still is time. As noted above, find me in the Social Media Lounge at the conference and in the sessions.  You can follow me on Twitter for conference updates @Writer1225.  I’ll be using hashtag #IBMEdge to post live Twitter comments from the conference. I’ll buy a drink for the first two people who come up to me and say they read DancingDinosaur.  How much more motivation do you need?

Next Generation zEnterprise Developers

April 19, 2013

Mainframe development keeps getting more complicated.  The latest complication can be seen in Doug Balog’s reference to mobile and social business on the zEnterprise, reported by DancingDinosaur here a few weeks ago. That is what the next generation of z developers face.

Forget talk about shortages of System z talent due to the retirement of mainframe veterans.  The bigger complication comes from need for non-traditional mainframe development skills required to take advantage mobile and social business as well as other recent areas of interest such as big data and analytics. These areas entail combining new skills like JSON, Atom, Rest, Hadoop, Java, SOA, Linux, hybrid computing along with traditional mainframe development skills like CICS and COBOL, z/VM, SQL, VSAM, and IMS. This combination is next to impossible to find in one individual. Even assembling a coherent team encompassing all those skills presents a serious challenge.

The mainframe industry has been scrambling to address this in various ways.  CA Technologies added GUI to its various tools and BMC has similarly modernized its various management and DB2 tools. IBM, of course, has been steadily bolstering the Rational RDz tool set.   RDz is a z/OS Eclipse-based software IDE.  RDz streamlines and refactors z/OS development processes into structured analysis, editing, and testing operations with modern GUI tools, wizards, and menus that, IBM notes, are perfect for new-to the-mainframe twenty- and thirty-something developers, the next generation of z developers.

Compuware brings its mainframe workbench, described as a modernized interactive developer environment that introduces a new graphical user interface for managing mainframe application development activities.  The interactive toolset addresses every phase of the application lifecycle.

Most recently, Micro Focus announced the release of its new Enterprise Developer for IBM zEnterprise.  The product enables customers to optimize all aspects of mainframe application delivery and promises to drive down costs, increase productivity, and accelerate innovation. Specifically, it enables both on- and off-mainframe development, the latter without consuming mainframe resources, to provide a flexible approach to the delivery of new business functions. In addition, it allows full and flexible customization of the IDE to support unique development processes and provides deep integration into mainframe configuration management and tooling for a more comprehensive development environment. It also boasts of improved application quality with measurable improvement in delivery times.  These capabilities together promise faster developer adoption.

Said Greg Lotko, Vice President and Business Line Executive, IBM System z, about the new Micro Focus offering:  We are continually working with our technology partners to help our clients maximize the value in their IBM mainframes, and this latest innovation from Micro Focus is a great example of that commitment.

Behind all of this development innovation is an industry effort to cultivate the next generation of mainframe developers. Using a combination of trusted technology (COBOL and mainframe) and new innovation (zEnterprise, hybrid computing, expert systems, and Eclipse), these new developers; having been raised on GUI and mobile and social, can leverage what they learned growing up to build the multi-platform, multi-device mainframe applications that organizations will need going forward.

As these people come on board as mainframe-enabled developers organizations will have more confidence in continuing to invest in their mainframe software assets, which currently amount to an estimated 200-300 billion lines of source code and may even be growing as mainframes are added in developing markets, considered a growth market by IBM.  It only makes sense to leverage this proven code base than try to replace it.

This was confirmed in a CA Technologies survey of mainframe users a year ago, which found that 1) the mainframe is playing an increasingly strategic role in managing the evolving needs of the enterprise; 2) the machine is viewed as an enabler of innovation as big data and cloud computing transform the face of enterprise IT—now add mobile; and 3) companies are seeking candidates with cross-disciplinary skill sets to fill critical mainframe workforce needs in the new enterprise IT thinking.

Similarly, a recent study by the Standish Group showed that 70 percent of CIOs saw their organizations’ mainframes as having a central and strategic role in their overall business success.  Using the new tools noted above organizations can maximize the value of the mainframe asset and cultivate the next generation mainframe developers.


Follow

Get every new post delivered to your Inbox.

Join 672 other followers

%d bloggers like this: