HPE has expanded its AI-at-scale offerings with the acquisition of Pachyderm, a start-up that delivers software, based on open-source technology, to automate reproducible machine learning pipelines that target large-scale AI applications.

With the addition of Pachyderm, HPE can unlock AI-at-scale opportunities for its customers by bringing together its leading supercomputing technologies that are foundational for optimized AI infrastructure, and the HPE Machine Learning Development Environment.

The combined solution already enables users to train more accurate AI models faster, and at scale, on supercomputers that have been purpose-built for demanding AI workloads.

HPE will integrate Pachyderm’s reproducible AI capabilities in one integrated platform to deliver an advanced data-driven pipeline that automatically refines, prepares, tracks, and manages repeatable machine learning processes used throughout the development and training environment.

Additionally, by integrating Pachyderm’s machine learning pipeline capabilities with its existing AI offerings, HPE can expect faster development and deployment of more accurate and performant large-scale AI applications, with benefits such as data lineage, data versioning, and efficient incremental data processing.

Justin Hotard, executive vice president and general manager, HPC and AI, at HPE, said: “As AI projects become larger and increasingly involve complex data sets, data scientists will need reproducible AI solutions to efficiently maximize their machine learning initiatives, optimize their infrastructure cost, and ensure data is reliable and safe no matter where they are in their AI journey.

“Pachyderm’s unique reproducible AI software augments HPE’s existing AI-at-scale offerings to automate and accelerate AI and unlock greater opportunities in image, video, and text analysis, generative AI, and other emerging large-language-model needs to realize transformative outcomes.”