Topic Model vs. Neural Network

Today, Joonhee and I discussed the neural network and the topic model at Starbucks. By the way, I highly recommend Starbucks’ new menu Green Tea Affogato Frappuccino, which is now available only in Korea.

Here the topic model indicates generative probabilistic models like LDA. The class of these models is called the “topic” model because this approach was first studied and employed mainly in document topic analyses, but now this approach is widely used in various domains and the models still seem to be called topic models by tradition. For the neural network, actually I’m not very knowledgable. What I learned in the machine learning class is all that I know. Anyhow, my opinion is this.

The topic model has several advantages over the neural network. When the results from the two models are compared, the topic model allows a clearer interpretation than the neural network. The neural network does not have explicit explanations of its process. Even though we analyze the mathematical process of each node and edge does in the model, their “roles” may not be human-interpretable. In contrast, the roles of the nodes and edges are explicitly modeled in the topic model (of course in bad models, the nodes and edges may act differently than expected), and thus it is relatively easy to interpret the result. This interpretability matter in turn affects the flexibility of the two models. When we get a bad result, for the topic model, we can find the part where unexpected behaviors occur, or we can change the previous assumptions and modify the model. For the neural network, however, it is not intuitive which nodes and edges to modify.

You may say it does not reflect the reality to assume a generation process of documents using probabilistic models, because these models are often simplified too much. The neural network could be more appropriate because the brain is indeed composed of neurons. Why not use the real implementation of the brain instead of the uncertain high-level and abstract probabilistic models? Another benefit of the neural network is the well-developed inference techniques such as back-propagation. On the contrary, different topic models usually require different inference processes and many times it is very tough to induce the right mathematical formula.

Again, I’m not really familiar with the neural network. If you visitors have opinions, your corrections and comments would be appreciated. Plus, I wonder if the neural network is still being widely used or it has become old-fashioned.

Topic Model vs. Neural Network