Update on Sept 05, 2016: I split-up the original blog (Part 1) into two to allow for easier reading.
The topic that started the discussion is a blog by Jeff Kyle, HPE Director of Mission Critical Systems promoting his Superdome X server at the expense, using straw men and simply factually incorrect information to base his arguments on.
Now it’s my turn to respond to all of Jeff’s FUD.
- Let’s start with this. My favorite topic right now which is to finally have an acknowledgement that Intel customers using VMware running Oracle Enterprise Edition Software products licensed by core have a problem.
- VMware’s Oracle problem is this. Oracle’s position is essentially if customers are running any Oracle Enterprise product licensed by core on a server running VMware, managed by vCenter then ALL (yes, ALL) servers in the cluster that are under that vCenter manager environment, should and MUST be licensed for ALL of the Oracle products running on that one server. Preposterous or not, it is not my fight. Obviously, VMware & Intel server vendors who are having their sales impacted by this are not happy. Oracle, which offers an x86 alternative in the form of Exadata and Oracle Database Appliance offer their own virtualization capabilities that is NOT VMware based and which clients do NOT have to license all of the cores, only those being used.
- VCE & House of Bricks, via a VCE sponsored whitepaper are encouraging customers to stand up to Oracle during contract negotiations, audits and in general to take the position that your business will only pay for the cores and thus the licenses with which are running Oracle Enterprise products. Of course, VCE, nor HoB nor any other Intel vendor that I have read about is providing any indemnification to customers who stand up to Oracle, found out of compliant with fines, penalties and fee’s. They have the choice to pay up or fight in court. Yes, it’s the right thing to do but keep in mind that Oracle is a company predisposed to litigate.
- Yes, I agree that Software licensing & maintenance costs are one of the largest cost items to a business. Far higher than infrastructure, yet Intel vendors wouldn’t have you believe that.
- IBM Power servers have several “forward looking Cloud deployment” technologies
- Open source products like PowerVC built on OpenStack manages an enterprise wide Power environment that integrates into a customer’s OpenStack environment.
- IBM Cloud PowerVC Manager, also built on OpenStack provides clients with enterprise wide private cloud capabilities.
- Both PowerVC and Cloud PowerVC Manager integrate with VMware’s vRealize allowing vCenter to manage a Power environment.
- If that isn’t enough, using IBM Cloud Orchestrator, clients can manage a heterogeneous compute & storage platform, both on-prem, hybrid or exclusively in the cloud.
- IBM will announce additional capabilities on September 8, 2016 that deliver more features to Power environments.
- “Proprietary chips” – so boring. What does that mean?
- Let’s look at Intel as they continue to close their own ecosystem. They bought an FPGA company with plans to integrate it into their silicon. Instead of working with GPU partners like NVIDIA & AMD, they developed their own GPU offering called Knights Landing. Long ago they integrated Ethernet controllers into their chips, and depending on the chip model, graphics capability. They build SSD’s, attempted to build mobile processors and my last example of them closing their ecosystem is Intel’s effort to build their own hi-speed, low latency communication transport called OmniPath instead of working with hi-speed Ethernet & InfiniBand partners like Mellanox. Of course, unlike Mellanox which provide offload processing capabilities on their adapters, true to form Intel’s OmniPath does not thus requiring them to use the main processing cores to service the hi-speed ethernet traffic. Wow, that should be some unpredictable utilization and increased core consumption…..which simply drives more servers and more licenses.
- Now let’s look at what Power is doing. IBM has opened up the Power Architecture specification to the OpenPOWER Foundation. Power.org is still the governing body but OpenPOWER is designed to encourage development & collaboration under a liberal license to build a broad ecosystem. Unlike Intel which is excluding more and more partners, OpenPOWER has partner companies building Power chips, systems, not to mention peripherals and software.
- I’ll spare you from looking up the definition of Open & Proprietary as I think it is clear who is really open and who is proprietary.
- Here is how the typical Intel “used car” salesman sells Oracle on x86: “Hey Steve, did you know that Oracle has a licensing factor of .5 on Intel while Power is 1.0? Yep, Power is twice as much. You know that means you will pay twice as much for your Oracle licenses! Hey, let’s go get a beer!”
- What Jeff is forgetting to tell you or simply does not know is that except for this unique example with Pella with Oracle running on a Superdome X server because of its nPAR capability, as most customers do not run Oracle on the larger Intel servers like that which may offer a hardware partitioning feature which allows for reduced licensing. They typically run it on 2 or 4 socket servers.
- The Superdome X server supports two types of partitioning that were carry overs from the original Superdome (Itanium) servers. vPARs and nPARs are both considered Hard Partitioning and thus, both allow the system to be configured into smaller groups of resources. This allows only those cores to be licensed, then adhering to Oracle licensing rules.
- HPE provides the Pella case study which states they have a 40 core partition separated from other cores on the server using nPAR technology that appears like server although made up of 2 blades. nPAR’s separate resources along “cell board” boundaries, which are the equivalent of an entire 2 socket blade. Pella’s Primary Oracle environment runs with 2 blades, each with 2 x 1o cores totaling 40 cores. These two production blades with 40 cores & 20 Oracle licenses sit alongside 2 other blades in one data center while at the failover site sits another HP SD-X chassis. I wonder if Pella realizes the inefficiency of the Superdome X solution. Every Intel server has a compromise. Traditional Scale-out 1 & 2 socket servers have compromises with scalability, performance & RAS. Traditional Scale-up 4 socket & larger Intel servers have compromises with scalability, performance and RAS as well. Each Superdome X blade has a Xbar controller plus the SX3000 Fabric bus. For this 4 socket NUMA server to act like one server, it will require 8 hops for every off-blade remote memory access. Further, if the 2nd blade isn’t in the same slot number scheme, such as blade 1 in slot 1 and blade 2 in slot 3, then performance would be further degraded. Do you see what I mean by Intel servers having land mines with every step?
- The Pella case study says the failover database server uses a single blade consisting of 30 cores. Not sure how they are doing that if they are using E7_v3 or E7_v4 processors as there is not a 15 core option. There is a E7_v2 (Ivy Bridge) 15 core option but doubt they would use it. This single Oracle DB failover blade sits with additional 2 blades. The fewest Oracle licenses you could pay for on the combined 4 socket or 40 core blade, assuming it is using 2 x 10 core chips per blade is 20 Oracle Licenses. So, even if the workload ONLY requires 8, 12, 16 or 18 cores the customer would still pay for 20 Licenses.
- This so-called $200,000 in Oracle licensing savings really is nothing, it really isn’t. I just showed a customer running Oracle EBS with Linux on Dell how they would save $2.5M per year in Oracle maintenance cost if they moved the workload to AIX on POWER8. If they would have deployed the solution on AIX to begin with, factoring in the 5 year TCO difference for what they are paying with Intel compared to POWER, this customer would have avoided spending $21M – let that sink in.
- I do not intend to be disrespectful to Pella, but if you would have put the Oracle workloads running on the older HP SuperDome onto POWER8 in 2015, you would not have bought a single Oracle license. I could guaranteed that you would have given licenses back, if desired. Not only would you avoid buying licenses, after returning licenses, you would save the 22% maintenance for each returned license.
- Look at one of my old blogs where I give some Oracle Licensing examples comparing licensing costs for Intel vs Power. It is typical for what I regularly see with clients, if not even greater consolidation ratio’s and subsequent license reductions results.
- The Pella case study does not mention if the new Superdome X solution uses any virtualization technology. I can only assume it does not since it was not mentioned. With IBM Power servers running AIX, all POWER servers come with virtualization (note I said “running AIX”). With Power, the customer could add/remove cores & memory. They could add & remove I/O to any LPAR (LPAR = VM) while doing concurrent maintenance on the I/O path out-of-band via dual VIOS, move that VM from one server to another live…maybe it is only used when upgrading to the next generation of server …. you know, POWER9; the next generation that would deliver to Pella even more performance per core, allowing them to return more Oracle licenses, saving even more money.
- This comes back to the “Granddaddy” statement Jeff made. Power servers have a license factor of 1.0 but with POWER server technology, customers ONLY license the cores used by Oracle. You can create dedicated VM’s where you only license those cores regardless of how many are in the server. Another option is to create a shared processor pool (SPP) and without going into all of the permutations, let’s simply say you ONLY license the cores used in the pool not to exceed the # of cores in the SPP. However, what is different from the dedicated VM is that within the SPP, there could be 1 – N VM’s sharing those cores and thus sharing the Oracle licenses.
- I did some analysis that I also use with my SAP HANA on POWER discussion where I show processor performance by core has increased each generation starting with POWER 5 all the way to POWER8. With POWER9 details just discussed at Hot Chips 2016 earlier this month (August), we can now expect it to deliver a healthy increase over POWER8 as well. Thus, P5 to P5+ saw an increase in per core performance. P5+ to P6 to P6+ to P7 to P7+ to P8 all saw successive increases in per core performance. Contrast that to Intel and reference a recent blog I wrote titled “Intel, the Great Charade”. Look at the first generation Nehalem called Gainestown which delivered a per core performance rating (as provided by Intel ) of .29. The next release was Westmere with a rating of .33. After that was Sandy Bride at .32 followed by Ivy Bridge at .31 then Haswell at .29 and the latest per core performance rating of .29. What does this mean? In 2016, the per core performance is the same as it was for a processor in the 2007 timeframe. Yes, they have more cores per socket – but I’ll ask you; how are you paying for your Oracle, by core or by socket?
- Next, IBM Power servers that run AIX, which is what would primarily run Oracle Enterprise Edition Database, run on servers with PowerVM which is the software suite that exploits Power Hypervisor. This is highly efficient and reliable firmware. Part of this efficiency is how it shares and thus dispatches unused processor cycles between VM’s not to mention the availability of 8 hardware threads per core, clock speeds up to 4.5 GHz, At least 2X greater L1 & L2 caches. 3.5X greater L3 cache and 100% more L4 cache over Intel. What does this mean? What it means is that Power does more than just beat Intel by 2X. That is what I call a “foot race”. When you factor in the virtualization efficiency you start to get processing efficiencies approaching 5:1, 10:1, even higher.
- I like to tell the story of a customer I had running Oracle EBS across two sites. It had additional Oracle products: RAC and WebLogic but this example will focus just on Location 1 and on Oracle Enterprise Edition Database. Customer was evaluating a Cisco UCS that was part of a vBlock, an Oracle Exadata and a IBM POWER7+ server. I’ll exclude Exadata, again because of some complexities it has with licensing where it skews the results against other Intel servers, just know the POWER7+ solution kicked its ass. With the vBlock, a VCE engineer sized the server & core requirements for the 2 site solution. Looking just at Location 1 for Oracle Enterprise Edition DB, the VCE engineer determined 300 Intel cores were required for Oracle EE DB. All of these workloads required varying degrees of cores; 7 cores in one server rounded up to 10. Another server required 4 cores that was rounded up to 6 or maybe 8 cores. Repeat this for dozens of workloads. Just to reiterate that VCE did the sizing as I did the POWER7+ sizing independent from VCE, completing mine first for that matter. My sizing demonstrated only 30 x POWER7+ cores. That was 300 Intel cores or 150 Oracle Licenses compared to 30 x POWER cores or 30 Oracle Licenses. If my memory serves me correctly, the hard core requirement for the Intel workload on the vBlock was around 168 cores which still would have been 84 Oracle Licenses. This customer was receiving a 75% discount from Oracle and even with this the difference in licensing cost (Oracle EE DB, RAC & WebLogic for 2 sites) was somewhere around $10-12M. Factor in the 22% annual maintenance and the 5 year TCO for the Intel solution ONLY for the Oracle software was around $20M vs $5-6M on POWER. By the way, the hardware solution cost differences were relatively close in price; within several $100K.
I know we are discussing Oracle on Intel but wanted to share this SAP 2-tier S&D performance comparison between 4, 8 and 16 socket Intel servers’ vs 2 & 8 socket POWER8 servers. I use this benchmark as I find it is one of the more reliable system wide test. Many of the benchmarks are focused in specific areas such as floating point or integer but not transactional data involving compute, memory movement and active I/O.
Note in the results the 4 socket Haswell server outperform the newer Broadwell 4 socket server. Next, notice the 8 socket Haswell server outperform the newer 8 socket Broadwell 8 socket server. Lastly, notice the 2 x 16 core results, both which are on a HP Superdome X server. Using the SAP benchmark measurement of SAPS, it shows the lowest amount of SAPS per core compared to any of the Intel servers shown. Actually, do you notice another pattern? The 4 sockets show greater efficiency over the 8 socket servers which show greater efficiency over the 16 socket servers.
Contrast that to the 2 socket POWER8 server, which by the way is 2X the best Intel result. If the trend we just reviewed with the Intel servers above holds true, we would expect the 8 socket POWER8 result to show fewer SAPS per core than the 2 socket POWER8 server. We already know the answer because it was highlighted in green as it was the highest result that was roughly 13% greater than the 2 socket POWER8. The 8 socket POWER8 was also 2.X+ greater than any of the Intel servers and 2.8X greater than the 16 socket HP Superdome X servers specifically.
Here comes my close – let’s see if I do a better job than Jeff!
- My last point is this in response to Jeff’s statement that “There’s a compelling alternative. A “scale-up” (high capacity) x86 architecture with a large memory capacity for in-memory compute models can dramatically improve database performance and lower TCO.”
- I’ve already debunked the myth and simply false statements that running Oracle on POWER costs more than Intel. In fact, it is just the opposite, and by a significant amount.
- Also, as shown in the HPE whitepaper “How memory RAS technologies can enhance the uptime of HPE ProLiant servers” they state “It might surprise you to know that memory device failures are far and away the most frequent type of failure for scale-up servers.”. It is amazing how HPE talks out of both sides of their mouth. Memory fails the most of any component in HPE servers yet they suggest you to buy these large scale-up servers that hold more memory, host and run more workloads such as “in-memory” from SAP HANA, Oracle 12c In-Memory or DB2 with BLU Acceleration. While in their own publishing’s they acknowledge it is the part most likely to fail in their solution.
- UPDATE: There is a better alternative to HPE Superdome X, Scale-up, Scale-out or any other Intel based server. That alternative has higher processor performance, larger memory bandwidth, a (much) more reliable memory subsystem as well as overall system RAS capabilities with a full suite of virtualization abilities. That alternative is an IBM Server, specifically POWER8 available in open source 1 & 2 socket configurations (look at LC models), scale-out 1 & 2 models & here (look a L models) and scale-up 4 – 16 socket Enterprise models. I’ll discuss more about HPE & IBM’s memory features in my next blog.
Your Honor, members of the jury, these are the facts as presented to you. I leave it to you to come back with the correct decision – Jeff Kyle and HPE are guilty of misleading customers and propagating untruths about IBM POWER.