Feature Engineering and Classifier Ensemble for KDD Cup 2010

Abstract

KDD Cup 2010 is an educational data mining competition. Participants are asked to learn a model from students ’ past behavior and then predict their future performance. At National Taiwan University, we organized a course for this competition. Most student sub-teams expanded features by various binarization and discretization techniques. The resulting sparse feature sets were trained by logistic regression (using LIBLINEAR). One sub-team considered condensed features using simple statistical techniques and applied Random Forest (through Weka) for training. Initial development was conducted on an internal split of training data for training and validation. We identified some useful feature combinations to improve performance. For the final submission, we combined results of student sub-teams by regularized linear regression. Our team is the first prize winner of both tracks (all teams and student teams) of KDD Cup 2010.

Keywords

Random forestArtificial intelligenceComputer scienceFeature engineeringMachine learningFeature (linguistics)Classifier (UML)Logistic regressionDiscretizationData miningPattern recognition (psychology)Deep learningMathematics

Affiliated Institutions

Related Publications

Fast and accurate image upscaling with super-resolution forests

Samuel Schulter , Christian Leistner , Horst Bischof

The aim of single image super-resolution is to reconstruct a high-resolution image from a single low-resolution input. Although the task is ill-posed it can be seen as finding a...

2015 657 citations

A survey on Image Data Augmentation for Deep Learning

Connor Shorten , Taghi M. Khoshgoftaar

Abstract Deep convolutional neural networks have performed remarkably well on many Computer Vision tasks. However, these networks are heavily reliant on big data to avoid overfi...

2019 Journal Of Big Data 11041 citations

Molecular Similarity Searching Using Atom Environments, Information-Based Feature Selection, and a Naïve Bayesian Classifier

Andreas Bender , Hamse Y. Mussa , Robert C. Glen +1 more

A novel technique for similarity searching is introduced. Molecules are represented by atom environments, which are fed into an information-gain-based feature selection. A naïve...

2003 Journal of Chemical Information and C... 246 citations

Kernel Logistic Regression and the Import Vector Machine

Ji Zhu , Trevor Hastie

The support vector machine (SVM) is known for its good performance in two-class classification, but its extension to multiclass classification is still an ongoing research issue...

2001 136 citations

Improving neural networks by preventing co-adaptation of feature detectors

Geoffrey E. Hinton , Nitish Srivastava , Alex Krizhevsky +2 more

When a large feedforward neural network is trained on a small training set, it typically performs poorly on held-out test data. This "overfitting" is greatly reduced by randomly...

2012 arXiv (Cornell University) 6630 citations

Publication Info

Year: 2010
Type: article
Citations: 128
Access: Closed

External Links

Citation Metrics

128

OpenAlex

Cite This

APA Style

                            
                                    Hsiang‐Fu Yu, 
                                
                                    Hung-Yi Lo, 
                                
                                    Hsun-Ping Hsieh
                                
                                et al.
                            
                            (2010). 
                            Feature Engineering and Classifier Ensemble for KDD Cup 2010. 
                            
                            .