High-Performance Computing (HPC)

High-Performance Computing, or HPC as it is generally known, has been a mainstay of infrastructure and hardware engineering groups for decades. Recent advancements in cloud computing have enabled a democratization of this once-specialized space that was restricted to hardware experts. Using proprietary parallel computing frameworks, RxDS has developed unique cluster-computing capabilities that can scale to more than 20,000+ AWS cores automatically, provision EC2 instances based on cost-based or performance-based instance selection, provide middleware connectivity, and much more.

Our flagship products, qPar and qIO, offer a wide range of performance and I/O functionalities for HPC Workloads. 

qPar and I/O

Further, qPar includes a comprehensive orchestration and scheduling layer that can automate scalable parallel workloads. The platform can run up to 20,000 tasks in parallel from a single node to reliably manage orchestration across thousands of independent tasks. Utilizing AWS spot instances delivers between 50-70% cost savings over running on-demand.

Simplicity of administration is achieved via a pre-built image. The administrative interface can capture task failures including stack trace from remote nodes. A user-friendly API for the Python, Matlab, and Julia programmer allows developers to utilize the capabilities from a familiar programming environment.

Recognizing that I/O is one of the common bottlenecks in data-heavy workloads, we have also developed our own I/O library. Known as qIO, it can deliver a peak speed of 15 Gigabytes per second of read on a single AWS instance.

qIO is built on strong technical foundations:

• Unique C-based library to communicate with the database
• No reliance of ODBC/JDBC drivers
• Complex datatypes handle via low-level APIs - direct memory access to both Julia database objects
• Configurable write-handlers and stream-handlers to efficiently support thousands of database clients