Hot on the heels of Google releasing an open source machine learning framework, Microsoft has released a similar project called DMLT (Distributed Machine Learning Toolkit). It simplifies machine learning work across distributed systems by allowing models to be trained (a core component of machine learning) on multiple nodes at once.
“Bigger models tend to generate better accuracies in various applications,” Microsoft writes in its introduction to the framework. “However, it remains a challenge for common machine learning researchers and practitioners to learn big models.”
DMLT’s core is a C++ SDK for a client-server architecture. “A number of server instances run on multiple machines and are responsible for maintaining the global model parameters,” says Microsoft in its documentation. “The training routines access and update the parameters with some client APIs that call the underlying communication facilities.”
Microsoft wants DMLT to make it easier for data scientists to perform model training across multiple machine nodes without having to worry about the nitty-gritty of managing threads or workloads. It simplifies interprocess communication, too, as two different libraries for that (MPI and ZMQ) are available and can be used interchangeably.
Two major algorithms for model training are included with DMLT. LightLDA will likely be the most commonly used for quick training of large data models.Microsoft claims it has been able to train models with “trillions of parameters” on only an eight-node system with LightLDA. Also included are Distributed Word Embedding and Distributed Multisense Word Embedding, algorithms for determining the relationships of words to each other.
DMLT was given an extremely low-key release by Microsoft. The only fanfare seems to have been blog posts on the DMLT site announcing that the source code had been made publicly available as of November 9. That said, Microsoft claims this is only the start of what it plans to offer with DMLT, as other algorithms are on the way.