A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization

Abstract

A probabilistic analysis of the Rocchio relevance feedback algorithm, one of the most popular learning methods from information retrieval, is presented in a text categorization framework. The analysis results in a probabilistic version of the Rocchio classifier and offers an explanation for the TFIDF word weighting heuristic. The Rocchio classifier, its probabilistic variant and a standard naive Bayes classifier are compared on three text categorization tasks. The results suggest that the probabilistic algorithms are preferable to the heuristic Rocchio classifier. This research is sponsored by the Wright Laboratory, Aeronautical Systems Center, Air Force Materiel Command, USAF, and the Advanced Research Projects Agency (ARPA) under grant F33615-93-1-1330. The US Government is authorized to reproduce and distribute reprints for Government purposes, notwithstanding any copyright notation thereon. Views and conclusions contained in this document are those of the authors and should not be ...

Keywords

Computer scienceProbabilistic logictf–idfArtificial intelligenceCategorizationClassifier (UML)Naive Bayes classifierText categorizationWeightingRelevance feedbackPattern recognition (psychology)Statistical classificationMachine learningNatural language processingMedicineSupport vector machine

Related Publications

HMM-based passage models for document classification and ranking

Ludovic Denoyer , Hugo Zaragoza , Patrick Gallinari

We present an application of Hidden Markov Models to supervised document classification and ranking. We consider a family of models that take into account the fact that relevant...

2001 40 citations

A comparison of event models for naive bayes text classification

Andrew McCallum , Kamal Nigam

Recent work in text classification has used two different first-order probabilistic models for classification, both of which make the naive Bayes assumption. Some use a multi-va...

1998 3224 citations

Machine learning in automated text categorization

Fabrizio Sebastiani

The automated categorization (or classification) of texts into predefined categories has witnessed a booming interest in the last 10 years, due to the increased availability of ...

2002 ACM Computing Surveys 7805 citations

Using Maximum Entropy for Text Classification

Kamal Nigam , John Lafferty , Andrew McCallum

This paper proposes the use of maximum entropy techniques for text classification. Maximum entropy is a probability distribution estimation technique widely used for a variety o...

1999 756 citations

On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes

Andrew Y. Ng , Michael I. Jordan

We compare discriminative and generative learning as typied by logistic regression and naive Bayes. We show, contrary to a widelyheld belief that discriminative classiers are al...

2001 1881 citations

Publication Info

Year: 1997
Type: article
Pages: 143-151
Citations: 1265
Access: Closed

External Links

Citation Metrics

1265

OpenAlex

Cite This

APA Style

                            
                                    Thorsten Joachims
                                
                            (1997). 
                            A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization. 
                            
                            , 143-151.