Topic Model vs. Neural Network

Today, Joonhee and I discussed the neural network and the topic model at Starbucks. By the way, I highly recommend Starbucks’ new menu Green Tea Affogato Frappuccino, which is now available only in Korea.

Here the topic model indicates generative probabilistic models like LDA. The class of these models is called the “topic” model because this approach was first studied and employed mainly in document topic analyses, but now this approach is widely used in various domains and the models still seem to be called topic models by tradition. For the neural network, actually I’m not very knowledgable. What I learned in the machine learning class is all that I know. Anyhow, my opinion is this.

The topic model has several advantages over the neural network. When the results from the two models are compared, the topic model allows a clearer interpretation than the neural network. The neural network does not have explicit explanations of its process. Even though we analyze the mathematical process of each node and edge does in the model, their “roles” may not be human-interpretable. In contrast, the roles of the nodes and edges are explicitly modeled in the topic model (of course in bad models, the nodes and edges may act differently than expected), and thus it is relatively easy to interpret the result. This interpretability matter in turn affects the flexibility of the two models. When we get a bad result, for the topic model, we can find the part where unexpected behaviors occur, or we can change the previous assumptions and modify the model. For the neural network, however, it is not intuitive which nodes and edges to modify.

You may say it does not reflect the reality to assume a generation process of documents using probabilistic models, because these models are often simplified too much. The neural network could be more appropriate because the brain is indeed composed of neurons. Why not use the real implementation of the brain instead of the uncertain high-level and abstract probabilistic models? Another benefit of the neural network is the well-developed inference techniques such as back-propagation. On the contrary, different topic models usually require different inference processes and many times it is very tough to induce the right mathematical formula.

Again, I’m not really familiar with the neural network. If you visitors have opinions, your corrections and comments would be appreciated. Plus, I wonder if the neural network is still being widely used or it has become old-fashioned.

Topic Model vs. Neural Network

One thought on “Topic Model vs. Neural Network

  1. 내생각엔 일단 두 모델의 쓰임새 자체가 다르고 두 모델의 영역이 다르기 때문에 비교가 가능하지 않을꺼같은데;;
    굳이 비교를 하자면 frequentist vs bayesian의 문제로 비교될수 있을까?
    뉴럴넷에서 히든 노드를 많이 만들면 많이 만들수록 overfitting 되는걸로 알고있는데
    그 자체에 뭔가 TM처럼 생성프로세스가 있는게 아니라 각각의 노드가 필터같은 역할을 하는거니까. 사실 요즘은 뉴럴넷보다 SVM이 대세아닌가?
    여튼 사람들이 뉴럴넷을 왜 많이 안쓰고 SVM을 더 많이 쓰는지를 비교하는게 적절할듯?
    Bayesian neural network vs neural network 의 비교를 하는게 적절한 비교대상일지도..
    여튼 스타벅스가다니 된장남 ㄷㄷㄷ

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s