litlas
ログインアカウント作成

BNS feature scaling: an improved representation over tf-idf for svm text classification

著者: George Forman - ACM International Conference on Information and Knowledge Management, CIKM 2008 被引用: 135

In the realm of machine learning for text classification, TF-IDF is the most widely used representation for real-valued feature vectors. However, IDF is oblivious to the training class labels and naturally scales some features inappropriately. We replace IDF with Bi-Normal Separation (BNS), which has been previously found to be excellent at ranking words for feature selection filtering. Empirical evaluation on a benchmark of 237 binary text classification tasks shows substantially better accuracy and F-measure for a Support Vector Machine (SVM) by using BNS scaling. A wide variety of other feature representations were later tested and found inferior, as well as binary features with no scaling. Moreover, BNS scaling yielded better performance without feature selection, obviating the need for feature selection.

Text Categorization withSupport Vector Machines…Text Categorization with Support Vector Machines: Learning with Many Relevant FeaturesInductive LearningAlgorithms and…Inductive Learning Algorithms and Representations for Text CategorizationFeature Selection forUnbalanced Class…Feature Selection for Unbalanced Class Distribution and Naive BayesFoundations ofstatistical natural…Foundations of statistical natural language processingCentroid-Based DocumentClassification: Analysi…Centroid-Based Document Classification: Analysis and Experimental ResultsAn Extensive EmpiricalStudy of Feature…An Extensive Empirical Study of Feature Selection Metrics for Text ClassificationText categorization withmany redundant features…Text categorization with many redundant features: using aggressive feature selection to make SVMs competitive with C4.5Feature Generation forText Categorization…Feature Generation for Text Categorization Using World KnowledgeTackling concept driftby temporal inductive…Tackling concept drift by temporal inductive transferTraining linear SVMs inlinear timeTraining linear SVMs in linear timeData MiningData MiningData Mining: PracticalMachine Learning Tools…Data Mining: Practical Machine Learning Tools and TechniquesBoosting KNN textclassification accuracy…Boosting KNN text classification accuracy by using supervised term weighting schemesFeature shaping forlinear SVM classifiersFeature shaping for linear SVM classifiersAutomatic SatireDetection: Are You…Automatic Satire Detection: Are You Having a Laugh?Ambiguity measurefeature-selection…Ambiguity measure feature-selection algorithmQuadratic ProgrammingFeature SelectionQuadratic Programming Feature SelectionSingle pass textclassification by direc…Single pass text classification by direct feature weightingFeature sub-setselection metrics for…Feature sub-set selection metrics for Arabic text classificationAutomatic categorizationof diverse experimental…Automatic categorization of diverse experimental information in the bioscience literatureHigh-Dimensional FeatureSelection by…High-Dimensional Feature Selection by Feature-Wise Kernelized LassoCombining supervisedterm-weighting metrics…Combining supervised term-weighting metrics for SVM text classification with extended term representationAre they Our Brothers?Analysis and Detection…Are they Our Brothers? Analysis and Detection of Religious Hate Speech in the Arabic TwittersphereDeep learning andnetwork analysis…Deep learning and network analysis: Classifying and visualizing accident narratives in constructionBNS feature scaling: animproved representation…BNS feature scaling: an improved representation over tf-idf for svm text classification過去の参考文献中心の論文この論文を引用する論文古い新しい

ノードをクリックするとフォーカスを固定、空白をクリックすると本論文に戻ります。ホバーで一時的にプレビューできます。各ノードのページはタイトルから開けます。