Microsoft, Intel Pledge Apache Spark Support: Big Data Roundup


In today's threat landscape, the only way for you to disrupt attackers and protect your organizati

Microsoft, Databricks, and Intel were among those lining up in support of Spark at this week's Spark Summit. We've got the full story in our Big Data Roundup for the week ending June 12, 2016.

It's Apache Spark time in our Big Data Roundup for the week ending June 12. At the Spark Summit West 2016, vendors big and small made announcements supporting the real-time big data analytics platform. Microsoft is getting behind Spark with several of its products. Distribution company Databricks revealed general availability of its Community Edition. And Intel declared Spark to be at the center of the big data revolution.

Let's start with Microsoft. This week the company announced general availability of Apache Spark 1.6.1 for Azure HDInsight, and Power BI support for Spark Streaming. Azure HDInsight is Microsoft's answer to Hadoop in the Azure cloud. Based on Hortonworks Data Platform Hadoop distribution, the service deploys and provisions managed Apache Hadoop clusters in the Azure cloud, providing a framework designed to process, analyze, and report on big data.

Now, the company is adding Spark for HDInsight, and Microsoft says it's a popular service, being adopted in 50% of all new HDInsight clusters deployed.

"With GA, we are revealing improvements we've made to the service to make Spark hardened for the enterprise and easy for your users," wrote Oliver Chiu, a senior product marketing manager for big data and data warehousing at Microsoft, in a blog post. "This includes improvements to the availability, scalability, and productivity of our managed Spark service."

[ Find out about what IBM announced at the Spark Summit. Read IBM Unveils Data Science Experience Dev Environment. ]

Microsoft also said it worked with Hortonworks to add capabilities to the YARN resource manager. In addition, Redmond co-led Project Livy with Cloudera and other organizations to create an open source Apache licensed REST web service for managing long-running Spark contexts and submitting Spark jobs.

Microsoft said it will offer an integration between Spark and the Azure Data Lake Store to enable Spark to store and process data of any size. Microsoft plans to enable role-based data access at the storage level through integration of Spark and the Data Lake Store.

And, for data scientists specifically, Microsoft introdcued out-of-the-box integration with Jupyter data science notebooks.

Microsoft had something for business intelligence professionals and analysts as well. The company will offer integration with Power BI and other BI tools such as Tableau, SAP Lumira, and QlikView.

"This lets you build interactive visualizations over data of any size," Chui wrote. "In addition to the traditional dashboards, Power BI offers a streaming connector that has integration with Spark allowing you to publish real-time events from Spark Streaming directly to Power BI."

Databricks is the chief commercial distribution company behind Apache Spark and this week the company announced general availability of its Data Bricks Community Edition, a free version of its just-in-time data platform built on top of Apache Spark.

The company said in a statement that DCE is accessible to all users, making it easy and quick to learn Apache Spark without the need to deal with infrastructure concerns.

"This year we've seen explosive growth for the Apache Spark project and all signs indicate the pace will only accelerate as the community expands even more," said Matei Zaharia, cofounder and chief technology officer at Databrick, in the statement. "Databricks Community Edition has created an ideal environment for learning Apache Spark. Developers of all backgrounds can now use Databricks Community Edition to learn Spark and mitigate the acute Spark skills gap."

In her presentation at the Spark Summit, Ziya Ma, Intel's Director of big data technologies, shared a statement made by the company's No. 3 employee Andy Grove about how analytics would be the No. 1 one workload in the data center by the year 2020. "Analytics enriches people's lives," she said.

"We believe Spark is at the center of the analytics revolution," Ma told attendees at the Summit.

In keeping with her company's commitment to big data and analytics, Ma provided an update on Intel's Trusted Analytics Platform. She also discussed a new chip announcement, the Xeon Processor E7 V4 family, which she said provided a 7x performance improvement for Spark workloads when moving from the previous generation of Intel hardware to this new one.

0 Comment

Leave a Reply

Captcha image


  • 5300c769af79e

    NVIDIA Hosting Massive Holiday Sale on Android Games

    49 (special launch price), Gunslugs for $1, and a whole lot more.Other titles on sale include Star Wars: KOTOR, Exiles, Parallax, Ravensword: Shadowlands, Party Hard, Zombie Tycoon 2, and Space Grunts.
  • 5300c769af79e

    The ‘Pokémon Go’ phenomenon just got an angry man fired from his job

    Boy, Pokémon Go is really getting some folks into a right ol’ tizzy, with one guy reportedly having just lost his job following an online rant about how he couldn’t get his hands on the new smartphone game.Gamer Sonny Truyen had recently moved to Singapore – where Pokémon Go isn’t yet available – from Australia, where it is.
  • 5300c769af79e

    Nokia Not Stopping With the Nokia 6, More Announcements Set for MWC

    As posted onto the company’s page, Nokia will make “more announcements” at MWC on February 26, 2017.While Nokia was light on the details, it seems possible that the company has more Android devices to announce for others markets around the globe, possibly even one for the US market.
  • 5300c769af79e

    Samsung Galaxy S7 Edge

    Compare Similar ProductsCompare Samsung Galaxy S7 %displayPrice% Google Nexus 6P (Unlocked) %displayPrice% Apple iPhone 6s Plus (Verizon Wireless) %displayPrice% LG V10 (AT&T) %displayPrice% Samsung Galaxy Note 5 (Verizon Wireless) %displayPrice% Samsung Galaxy S6 edge+ (Verizon Wireless) %displayPrice% Samsung Galaxy S6 (Verizon Wireless) %displayPrice% Droid Turbo 2 by Motorola (Verizon Wireless) %displayPrice% Motorola Moto X Pure Edition (Unlocked) %displayPrice% HTC One M9 (T-Mobile) %displayPrice% OnePlus 2 (Unlocked) %displayPrice% DesignThe S7 Edge borrows ideas from a whole bunch of last year's Samsung phones, and puts them together in the right combination.It has the tight efficiency of the S6 Edge, a bigger screen like the slightly bloated S6 Edge+, and superior edge functionality from the older Galaxy Note Edge.