Calculate the average token length for each topic in a topic model

Source: R/mean_token_length.R

Using the the N highest probability tokens for each topic, calculate the average token length for each topic

mean_token_length(topic_model, top_n_tokens = 10)

Arguments

topic_model

a fitted topic model object from one of the following: tm-class

top_n_tokens

an integer indicating the number of top words to consider, the default is 10

Value

A vector of average token lengths with length equal to the number of topics in the fitted model

References

Jordan Boyd-Graber, David Mimno, and David Newman, 2014. Care and Feeding of Topic Models: Problems, Diagnostics, and Improvements. CRC Handbooks ofModern Statistical Methods. CRC Press, Boca Raton, Florida.

Examples


# Using the example from the LDA function
library(topicmodels)
data("AssociatedPress", package = "topicmodels")
lda <- LDA(AssociatedPress[1:20,], control = list(alpha = 0.1), k = 2)
mean_token_length(lda)
#> [1] 6 5