- Twitter is moving less-used cold data storage systems and a portion of its key Hadoop compute clusters to the Google Cloud Platform.
- Google Cloud Platform’s data solutions and trusted infrastructure will provide Twitter with the technical flexibility and consistency that it requires.
While the majority of data processing and storage capacity will remain in Twitter data centers, the migration to Google is seen as a good mix-and-match approach to stay ahead of the game.
Twitter is moving parts of its data storage and processing system to the Google cloud to increase efficiency. San-Francisco-headquartered Twitter is moving less-used cold data storage systems and a portion of its key Hadoop compute clusters to the Google Cloud Platform. Twitter’s key data centers, where these systems are currently housed, are in the US.
Speaking of the decision to move systems to Google, Twitter CTO Parag Agrawal said: “It is critical that our infrastructure and data platforms are able to scale. Over the past few years, we have been assessing our platform and infrastructure needs to make sure we are well positioned to keep up with the growing needs of our service.”
He said the Hadoop compute system is the “core” of Twitter’s data platform, with the company running multiple large Hadoop clusters that are among the biggest in the world. These file systems host more than 300 petabytes of data across tens of thousands of servers.
Want to publish your own articles on DistilINFO Publications?
Send us an email, we will get in touch with you.
“We are working with Google Cloud to move cold data storage and our flexible compute Hadoop clusters to the Google Cloud Platform. This will enable us to enhance the experience and productivity of our engineering teams working with our data platform,” said Agrawal.
When complete, said Agrawal, the migration will enable faster capacity provisioning, increased flexibility, access to a broader ecosystem of tools and services, improvements to security and enhanced disaster recovery capabilities.
He added: “Architecturally, we will also be able to separate compute and storage for these Hadoop workloads, which has a number of long-term scaling and operational benefits.”
Brian Stevens, CTO of Google Cloud, said of the deal: “Google Cloud Platform’s data solutions and trusted infrastructure will provide Twitter with the technical flexibility and consistency that it requires.”
Date: May 08, 2018