Klimasauskas Group^TM
Leading the Edge in Emerging Technologies

Neural Networks are a technology that grew out of studies of the brain and nervous system. Some of the earliest work with Neural Networks involved modeling different kinds of behaviors. Behavior modeling continues to be a significant application area of Neural Networks. In particular, NeuralWare's NeuralWorks Professional II/Plus is often the tool of choice.

Automating Work Processes & Improving Network Performance on Small Data Sets. One recent project that staff of Klimasauskas Group worked on was to improve the work-processes and network consistency in a behavioral modeling application. This particular customer was developing models with small data sets (typically less than 100 examples), and many inputs (typically more than 25). Their data was in Excel, and they were using NeuralWorks Professional II/Plus for their neural network development. As part of their validation process, they would do an n-fold cross-validation as a way of estimating the reliability of their final prediction on a single unknown case. There were two challenges that they faced. First, the entire process required many manual steps and typically took 5-6 hours. Second, due to the small data set size and number of inputs, multiple runs (with different randomizations) resulted in small discrepancies between predictions from run-to-run.

The solution was to develop a set of Excel Macros (in Visual Basic) that controlled the automation of all of the manual procedures. In support of this, the batch training capability of NeuralWorks Professional II/Plus was extended. In order to address the consistency of prediction, the networks were "annealed", a technique borrowed from Scott Kirkpatrick's Simulated Annealing. This was implemented through the NeuralWorks' User IO capability. All together, the integrated work processes reduced network development time from 6 hours to 20 minutes, and significantly improved consistency of predictions.

Monte Carlo Simulation for Estimating Network Performance. Have you every wondered whether your Neural Network predictions are better than chance? In many applications, particularly where the input data may be uncertain, the only solution is to simulate the kinds of outcomes that might be achieved by chance. As an example, in one prediction problem, an accuracy of 80% was exceeded by chance 40% of the time. However, an accuracy of 85% was exceeded only 2% of the time (subject to prior probabilities and structural constraints). The only way to know if your results are significant is by simulation.

Key Benefits

• Improved / automated work processes reducing network training time from 6 hours to 20 minutes.
• Improved consistency in predictions resulting in higher confidence in one crucial prediction.
• Reduced errors. Extensive data consistency checking substantially reduced errors.
• New opportunities and areas for further research were opened up by this capability.
• Statistical validation of network performance through simulation.

Capabilities

Klimasauskas Group staff have extensive experience in developing Neural Network technology, and working with the integration of existing products such as NeuralWorks Professional II/Plus. In this project, we were able to leverage our understanding of Neural Network learning mechanisms to improve overall consistency.

• Application specific solutions for improving neural network prediction performance
• Excel and Visual Basic application development
• NeuralWorks Professional II/Plus integration
• Monte Carlo simulation for estimating significance of network performance.