Apache Spark and Cloudera Data Platform are key players in the big data landscape, excelling in different areas. Apache Spark leads with high-performance analytics through its rapid processing capabilities, whereas Cloudera Data Platform offers more comprehensive data management and security features.
Features: Apache Spark is renowned for its fast in-memory data processing, efficiency in handling batch and stream processing, and support for multiple programming languages like Scala, Java, and Python. Cloudera Data Platform shines with integrated data management, robust security features, and a comprehensive suite of analytics tools.
Room for Improvement: Apache Spark could enhance its data governance capabilities, improve integration with data management tools, and bolster support for enterprise-level security features. Cloudera Data Platform might improve its ease of use, enhance compatibility with a broader range of tools, and offer more flexibility in data processing options to meet diverse needs.
Ease of Deployment and Customer Service: Apache Spark is easy to deploy across various environments with a straightforward setup process. However, Cloudera Data Platform offers superior customer support, providing tailored solutions ideal for enterprise needs, making it favorable for organizations needing comprehensive support in deployment.
Pricing and ROI: Apache Spark being open-source ensures low setup costs and high ROI due to its performance efficiency. In contrast, Cloudera Data Platform may have higher initial costs, but offers better integration capabilities and data governance tools, potentially providing greater ROI over time by improving productivity in large-scale operations.
I have communicated with technical support, and they are responsive and helpful.
Integration with other tools works well for us and we successfully scaled the solution after two to three years without any issues.
For scalability, I rate Cloudera Data Platform at an eight out of ten as it is an on-premise solution.
We aim to address these issues with a Kubernetes-based platform that will simplify the task of upgrading services.
Cloudera Data Platform should include additional capabilities and features similar to those offered by other data management solutions like Azure and Databricks.
Initially, CDH had a straightforward pricing model based on nodes, but CDP includes factors like processors, cores, terabytes, and drives, making it difficult to calculate costs.
By using the Hadoop File System for distributed storage, we have 1.5 petabytes of physical storage with 500 terabytes of effective storage due to a replication factor of three.
The foremost benefit is offloading data from the warehouse to Cloudera Data Platform, which allows for cheaper storage.
Spark provides programmers with an application programming interface centered on a data structure called the resilient distributed dataset (RDD), a read-only multiset of data items distributed over a cluster of machines, that is maintained in a fault-tolerant way. It was developed in response to limitations in the MapReduce cluster computing paradigm, which forces a particular linear dataflowstructure on distributed programs: MapReduce programs read input data from disk, map a function across the data, reduce the results of the map, and store reduction results on disk. Spark's RDDs function as a working set for distributed programs that offers a (deliberately) restricted form of distributed shared memory
Cloudera Data Platform offers a powerful fusion of Hadoop technology and user-centric tools, enabling seamless scalability and open-source flexibility. It supports large-scale data operations with tools like Ranger and Cloudera Data Science Workbench, offering efficient cluster management and containerization capabilities.
Designed to support extensive data needs, Cloudera Data Platform encompasses a comprehensive Hadoop stack, which includes HDFS, Hive, and Spark. Its integration with Ambari provides user-friendliness in management and configuration. Despite its strengths in scalability and security, Cloudera Data Platform requires enhancements in multi-tenant implementation, governance, and UI, while attribute-level encryption and better HDFS namenode support are also needed. Stability, especially regarding the Hue UI, financial costs, and disaster recovery are notable challenges. Additionally, integration with cloud storage and deployment methods could be more intuitive to enhance user experience, along with more effective support and community engagement.
What are the key features?Cloudera Data Platform is implemented extensively across industries like hospitality for data science activities, including managing historical data. Its adaptability extends to operational analytics for sectors like oil & gas, finance, and healthcare, often enhanced by Hortonworks Data Platform for data ingestion and analytics tasks.
We monitor all Hadoop reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.