FROM DAYS TO MINUTES - Improving Performance with Machine Learning
The console output below illustrates the potential advantages of machine learning approaches when used appropriately and sparingly. The computation to be performed is extremely complex and costly to evaluate. By first implementing heuristics derived from domain-specific knowledge in the form of sampling-based methodologies and data preprocessing, we significantly reduced complexity. This approach was sufficient to provide initial results in a reasonable time. However, it became increasingly infeasible (~45 hours computation time) as model complexity increased, reflecting the state-of-the-art real-world data. This is because the underlying space is incredibly large, and the number of computations scales exponentially with each dimension.
After thoroughly analyzing the underlying space, we decided to use a #machinelearning technique to leverage its rather simplistic and well-behaved nature. This allowed us to effectively learn its composition using only a small but carefully selected fraction of precomputed data with remarkably high accuracy (98%), thus substituting most of the costly computations and reducing the evalutation time from days to minutes.
UNDERSTANDING YOUR TOOLBOX
The conclusion is that there is enormous potential for performance improvement in industrial settings using what is generally called "AI". However, to effectively leverage its capability, a deep understanding of the underlying problem is necessary. Like any other method, #AI is simply a tool—a powerful one, sure—but still a tool. In the aforementioned example taken from our own R&D-project at CORE MUC GmbH , the performance improvement would not have been possible without the prior engineering work, which provided a solid foundation for the efficient and effective use of machine learning.