14.1.14 Classifier, Performance Evaluation, Errors, Comparisons

Mattson, R.L., and Firschein, O.,
Feature Word Construction for Use with Pattern Recognition Algorithms: An Experimental Study,
JACM(10), No. 4, October 1963, pp. 458-477. BibRef 6310

Dressler, R.F., and Werner, W.,
Error Rates for Two Methods of Statistical Pattern Recognition,
JACM(11), No. 4, October 1964, pp. 471-480. BibRef 6410

Bledsoe, W.W.,
Some Results on Multicategory Pattern Recognition,
JACM(13), No. 2 April 1966, pp. 304-316. BibRef 6604

Hughes, G.F.,
On the Mean Accuracy of Statistical Pattern Recognizers,
IT(14), No. 1, January 1968, pp. 55-63. BibRef 6801

Chow, C.K.,
On Optimum Recognition Error and Reject Trade-Offs,
IT(16), No. 1, January 1970, pp. 41-46. Relationship between (False Rejection) FRR and (False Acceptance) FAR error formulations.
See also Optimum Character Recognition System Using Decision Function, An. BibRef 7001

Rand, W.M.,
Objective Criteria for the Evaluation of Clustering Methods,
ASAJ(66), No. 336, 1971, pp. 846-850. BibRef 7100

Preston, K.,
A Comparison of Analog and Digital Techniques for Pattern Recognition,
PIEEE(60), No. 10, October 1972, pp. 1216-1231. BibRef 7210

Sacco, W.J.[William J.], Copes, W.S.[Wayne S.],
Reduction of the class of feature evaluation techniques in pattern analysis,
PR(4), No. 3, October 1972, pp. 331-332.
Elsevier DOI 0309
Letter. BibRef

Toussaint, G.T.[Godfried T.],
Bibliography on Estimation of Misclassification,
IT(20), 1974, pp. 472-479. Survey, Evaluation. Complete listing of the research up to this time. BibRef 7400

Slagle, J.R., Chang, C.L., Lee, C.T.,
Experiments with Some Cluster Analysis Algorithms,
PR(6), No. 3-4, December 1974, pp. 181-187.
Elsevier DOI 1-D clustering. BibRef 7412

Milligan, G.W.[Glenn W.], Isaac, P.D.[Paul D.],
The validation of four ultrametric clustering algorithms,
PR(12), No. 2, 1980, pp. 41-50.
Elsevier DOI 0309
BibRef

Bayne, C.K.[Charles K.], Beauchamp, J.J.[John J.], Begovich, C.L.[Connie L.], Kane, V.E.[Victor E.],
Monte Carlo comparisons of selected clustering procedures,
PR(12), No. 2, 1980, pp. 51-62.
Elsevier DOI 0309
BibRef

Viscolani, B.[Bruno],
Computational length in pattern recognizers,
PR(15), No. 5, 1982, pp. 413-418.
Elsevier DOI 0309
BibRef

Viscolani, B.[Bruno],
Optimization of computational time in pattern recognizers,
PR(15), No. 5, 1982, pp. 419-424.
Elsevier DOI 0309
A sequential organization of the computations arising from pattern recognizers by absolute comparison is suggested in order to reduce the mean computational time involved. BibRef

Viscolani, B.[Bruno],
Further results on optimization of recognition time,
PR(16), No. 3, 1983, pp. 337-339.
Elsevier DOI 0309
BibRef

Hanley, J., McNeil, B.,
The Meaning and Use of the Area under a Receiver Operating Characteristic (ROC) Curve,
Radiology(143), 1982, pp. 29-36. BibRef 8200

Goin, J.E.[James E.], Fritz, S.L.[Steven L.],
A Matrix Approach to Data Base Exploration: Analysis of Classifier Results,
PR(16), No. 2, 1983, pp. 243-252.
Elsevier DOI 0309
BibRef

Raveh, A.[Adi],
Preference structure analysis: A nonmetric approach,
PR(16), No. 2, 1983, pp. 253-259.
Elsevier DOI 0309
Suggest a nonmetric procedure in which goodness of discrimination is higher than or equal to that of Fisher's discriminant function. very different discriminant functions could yield the very same number of errors. BibRef

Fowlkes, E.B., and Mallows, C.L.,
A Method for Comparing Two Hierarchical Clusterings,
ADAJ(78), No. 383, 1983, pp. 553-569. BibRef 8300

Flick, T.E., and Jones, L.K.,
A Combinatorial Approach for Classification of Patterns with Missing Information and Random Orientation,
PAMI(8), No. 4, July 1986, pp. 482-490. BibRef 8607

Flick, T.E.[Thomas E.], Jones, L.K.[Lee K.], Priest, R.G.[Richard G.], Herman, C.[Charles],
Pattern classification using projection pursuit,
PR(23), No. 12, 1990, pp. 1367-1376.
Elsevier DOI 0401
BibRef

Wacker, A.G., El-Sheikh, T.S.,
Average Classification Accuracy over Collections of Gaussian Problems: Common Covariance Matrix Case,
PR(17), No. 2, 1984, pp. 259-273.
Elsevier DOI 0309
BibRef

Chernick, M.C., Murthy, V.K., and Nealy, C.D.,
Application of Bootstrap and Other Resampling Techniques: Evaluation of Classifier Performance,
PRL(3), 1985, pp. 167-178. BibRef 8500

Jain, A.K., Dubes, R.C., and Chen, C.C.,
Bootstrap Techniques for Error Estimation,
PAMI(9), No. 5, September 1987, pp. 628-633. BibRef 8709

Colussi, L.,
Correctness and Efficiency of Pattern Matching Algorithms,
InfoControl(95), 1991, pp. 225-251. BibRef 9100

Srivastava, A.[Anurag], Murty, M.N.,
A comparison between conceptual clustering and conventional clustering,
PR(23), No. 9, 1990, pp. 975-981.
Elsevier DOI 0401
BibRef

Tang, Y.Y., Qu, Y.Z., Suen, C.Y.,
Multiple-level information source and entropy-reduced transformation models,
PR(24), No. 4, 1991, pp. 341-357.
Elsevier DOI 0401
Analyze systematically the changes in entropy which occur in the different stages of a pattern recognizer. BibRef

Gluhchev, G., Shalev, S.,
The Systematic-Error Detection as a Classification Problem,
PRL(17), No. 12, October 25 1996, pp. 1233-1238. 9612
BibRef

Duin, R.P.W.,
A Note on Comparing Classifiers,
PRL(17), No. 5, May 1 1996, pp. 529-536. 9606
BibRef

Richards, J.A.,
Classifier Performance and MAP Accuracy,
RSE(57), No. 3, September 1996, pp. 161-166. 9609
BibRef

Nyssen, E.,
Evaluation of Pattern Classifiers: Testing the Significance of Classification Efficiency Using an Exact Probability Technique,
PRL(17), No. 11, September 16 1996, pp. 1125-1129. 9611
BibRef

Nyssen, E.,
Evaluation of Pattern Classifiers: Applying a Monte Carlo Significance Test to the Classification Efficiency,
PRL(19), No. 1, January 1998, pp. 1-6. 9807
BibRef

San Miguel-Ayanz, J., Biging, G.S.,
Comparison of Single-Stage and Multistage Classification Approaches for Cover Type Mapping with TM and Spot Data,
RSE(59), No. 1, January 1997, pp. 92-104. 9701
BibRef

Denoeux, T.[Thierry],
Analysis of Evidence Theoretic Decision Rules for Pattern-Classification,
PR(30), No. 7, July 1997, pp. 1095-1107.
Elsevier DOI 9707
BibRef

Bokka, V., Olariu, S., Schwing, J.L., Wilson, L., Zomaya, A.,
A Time-Optimal Solution to a Classification Problem in Ordered Functional Domains, with Applications,
PR(30), No. 9, September 1997, pp. 1555-1564.
Elsevier DOI 9708
BibRef

Zhou, J.Y.[Jiang-Ying], Lopresti, D.P.[Daniel P.],
Improving Classifier Performance Through Repeated Sampling,
PR(30), No. 10, October 1997, pp. 1637-1650.
Elsevier DOI 9712
BibRef

Lerner, B.[Boaz], Guterman, H.[Hugo], Aladjem, M.[Mayer], Dinsteint, I.[Its'hak], Romem, Y.[Yitzhak],
On Pattern Classification with Sammons Nonlinear Mapping: An Experimental-Study,
PR(31), No. 4, April 1998, pp. 371-381.
Elsevier DOI 9803

See also nonlinear mapping for data structure analysis, A. BibRef

Aladjem, M.[Mayer], Dinstein, I.[Its'hak],
A multiclass extension of discriminant mappings,
ICPR92(II:101-104).
IEEE DOI 9208
BibRef

Stehman, S.V., Czaplewski, R.L.,
Design and Analysis for Thematic Map Accuracy Assessment: Fundamental Principles,
RSE(64), No. 3, June 1998, pp. 331-344. 9806
BibRef

Adams, N.M., Hand, D.J.,
Comparing classifiers when the misallocation costs are uncertain,
PR(32), No. 7, July 1999, pp. 1139-1147.
Elsevier DOI BibRef 9907

Smits, P.C., Dellepiane, S.G., Schowengerdt, R.A.,
Quality assessment of image classification algorithms for land-cover mapping: a review and a proposal for a cost-based approach,
JRS(20), No. 8, May 1999, pp. 1461. BibRef 9905

Sohn, S.Y.[So Young],
Meta Analysis of Classification Algorithms for Pattern Recognition,
PAMI(21), No. 11, November 1999, pp. 1137-1144.
IEEE DOI 9912
For sample size and dimensionality. Meta model to compare different classification algorithms. Traditional statistical, neural nets, and machine learning approaches. BibRef

Srivastava, A.N., Su, R., Weigend, A.S.,
Data Mining for Features Using Scale-Sensitive Gated Experts,
PAMI(21), No. 12, December 1999, pp. 1268-1279.
IEEE DOI 0001
Data analysis to partition complex regression surface into simpler surfaces (features).
See also Virtual Sensors: Using Data Mining Techniques to Efficiently Estimate Remote Sensing Spectra. BibRef

Andersson, A.[Arne], Davidsson, P.[Paul], Lindén, J.[Johan],
Measure-based classifier performance evaluation,
PRL(20), No. 11-13, November 1999, pp. 1165-1173. 0001
BibRef

Lim, T.S., Loh, W.Y., Shil, Y.S.,
A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms,
MachLearn(40), No. 3, 2000, pp. 203-228. 0003
BibRef

Ong, S.H., Zhao, X.,
On post-clustering evaluation and modification,
PRL(21), No. 5, May 2000, pp. 365-373. 0005
BibRef

Raudys, S.J.[Sarunas J.], Saudargiene, A.[Ausra],
First-Order Tree-Type Dependence between Variables and Classification Performance,
PAMI(23), No. 2, February 2001, pp. 233-239.
IEEE DOI 0102
BibRef

Hubert-Moy, L., Cotonnec, A., Le Du, L., Chardin, A., Perez, P.,
A Comparison of Parametric Classification Procedures of Remotely Sensed Data Applied on Different Landscape Units,
RSE(75), No. 2, 2001, pp. 174-187. 0102
BibRef

Tambouratzis, G.[George],
Improving the Clustering Performance of the Scanning n-Tuple Method by Using Self-Supervised Algorithms to Introduce Subclasses,
PAMI(24), No. 6, June 2002, pp. 722-733.
IEEE DOI 0206
BibRef
Earlier:
Improving the Classification Accuracy of the Scanning N-tuple Method,
ICPR00(Vol II: 1046-1049).
IEEE DOI 0009
Extend work of:
See also Statistical Syntactic Methods for High-Performance OCR. Remove edge effects. BibRef

Liu, M.Q.[Ming-Qin], Samal, A.[Ashok],
Cluster validation using legacy delineations,
IVC(20), No. 7, May 2002, pp. 459-467.
Elsevier DOI 0206
BibRef

Berikov, V.B.[Vladimir B.],
An approach to the evaluation of the performance of a discrete classifier,
PRL(23), No. 1-3, January 2002, pp. 227-233.
Elsevier DOI 0201
BibRef

Harvey, N.R., Theiler, J., Brumby, S.P., Perkins, S., Szymanski, J.J., Bloch, J.J., Porter, R.B., Galassi, M., Young, A.C.,
Comparison of GENIE and conventional supervised classifiers for multispectral image feature extraction,
GeoRS(40), No. 2, February 2002, pp. 393-404.
IEEE Top Reference. 0205
BibRef

Muchoney, D.M.[Douglas M.], Strahler, A.H.[Alan H.],
Pixel- and site-based calibration and validation methods for evaluating supervised classification of remotely sensed data,
RSE(81), No. 2-3, August 2002, pp. 290-299.
HTML Version. 0206
BibRef

Alsing, S.G.[Stephen G.], Bauer, Jr., K.W.[Kenneth W.], Miller, J.O.[John O.],
A multinomial selection procedure for evaluating pattern recognition algorithms,
PR(35), No. 11, November 2002, pp. 2397-2412.
Elsevier DOI 0208
BibRef

Maulik, U.[Ujjwal], Bandyopadhyay, S.[Sanghamitra],
Performance Evaluation of Some Clustering Algorithms and Validity Indices,
PAMI(24), No. 12, December 2002, pp. 1650-1654.
IEEE Abstract. 0212
Hard K-Means, Single Linkage, Simulated annealing
See also Optimization by Simulated Annealing. BibRef

Sandri, L.[Laura], Marzocchi, W.[Warner],
Testing the performance of some nonparametric pattern recognition algorithms in realistic cases,
PR(37), No. 3, March 2004, pp. 447-461.
Elsevier DOI 0401
BibRef

Toh, K.A.[Kar-Ann], Tran, Q.L.[Quoc-Long], Srinivasan, D.[Dipti],
Benchmarking a Reduced Multivariate Polynomial Pattern Classifier,
PAMI(26), No. 6, June 2004, pp. 740-755.
IEEE Abstract. 0404
The simplified model worked well. Analyze it. BibRef

Tran, Q.L.[Quoc-Long], Toh, K.A.[Kar-Ann], Srinivasan, D.[Dipti], Wong, K.L., Low, S.Q.C.[Shaun Qiu-Cen],
An empirical comparison of nine pattern classifiers,
SMC-B(35), No. 5, October 2005, pp. 1079-1091.
IEEE DOI 0510
Algorithm RM. Reduced Multivariate. BibRef

Attoor, S.N.[Sanju N.], Dougherty, E.R.[Edward R.],
Classifier performance as a function of distributional complexity,
PR(37), No. 8, August 2004, pp. 1641-1651.
Elsevier DOI 0407
BibRef

Kim, D.W.[Dae-Won], Lee, K.Y.[Ki Young], Lee, D.[Doheon], Lee, K.H.[Kwang H.],
Evaluation of the performance of clustering algorithms in kernel-induced feature space,
PR(38), No. 4, April 2005, pp. 607-611.
Elsevier DOI 0501
BibRef

Kim, D.W.[Dae-Won], Lee, K.Y.[Ki-Young], Lee, D.[Doheon], Lee, K.H.[Kwang H.],
A k-populations algorithm for clustering categorical data,
PR(38), No. 7, July 2005, pp. 1131-1134.
Elsevier DOI 0505
BibRef

Stein, A., Aryal, J., Gort, G.,
Use of the Bradley-Terry Model to Quantify Association in Remotely Sensed Images,
GeoRS(43), No. 4, April 2005, pp. 852-856.
IEEE Abstract. 0501
BibRef

Caulfield, H.J.[H. John], Heidary, K.[Kaveh],
Exploring margin setting for good generalization in multiple class discrimination,
PR(38), No. 8, August 2005, pp. 1225-1238.
Elsevier DOI 0505
BibRef

Salman, A.[Ayed], Omran, M.G.[Mahamed G.], Engelbrecht, A.P.[Andries P.],
SIGT: Synthetic Image Generation Tool for Clustering Algorithms,
GVIP(05), No. V2, January 2005, pp. 33-44
HTML Version. Create images to test clustering. BibRef 0501

Graaff, A.J., Engelbrecht, A.P.[Andries P.],
Clustering data in an uncertain environment using an artificial immune system,
PRL(32), No. 2, 15 January 2011, pp. 342-351.
Elsevier DOI 1101
Uncertain environments; Non-stationary data; Immune networks; Clustering performance measures BibRef

Yousef, W.A.[Waleed A.], Wagner, R.F.[Robert F.], Loew, M.H.[Murray H.],
Estimating the uncertainty in the estimated mean area under the ROC curve of a classifier,
PRL(26), No. 16, December 2005, pp. 2600-2610.
Elsevier DOI 0512
BibRef

Yousef, W.A.[Waleed A.], Wagner, R.F.[Robert F.], Loew, M.H.[Murray H.],
Assessing Classifiers from Two Independent Data Sets Using ROC Analysis: A Nonparametric Approach,
PAMI(28), No. 11, November 2006, pp. 1809-1817.
IEEE DOI 0609
BibRef
Earlier:
Comparison of non-parametric methods for assessing classifier performance in terms of ROC parameters,
AIPR04(190-195).
IEEE DOI 0410
3 Parameters: Conditional (an particular training set) AUC (area under RO Curve), mean and variance of AUC. BibRef

Baraldi, A., Bruzzone, L., Blonda, P., Carlin, L.,
Badly Posed Classification of Remotely Sensed Images: An Experimental Comparison of Existing Data Labeling Systems,
GeoRS(44), No. 1, January 2006, pp. 214-235.
IEEE DOI 0601
BibRef

Baraldi, A., Bruzzone, L., Blonda, P.,
A Multiscale Expectation-Maximization Semisupervised Classifier Suitable for Badly Posed Image Classification,
IP(15), No. 8, August 2006, pp. 2208-2225.
IEEE DOI 0606
BibRef

Fawcett, T.[Tom],
An introduction to ROC analysis,
PRL(27), No. 8, June 2006, pp. 861-874.
Elsevier DOI 0605
Survey, ROC Analysis. Classifier evaluation; Evaluation metrics BibRef

Stathakis, D., Vasilakos, A.,
Comparison of Computational Intelligence Based Classification Techniques for Remotely Sensed Optical Image Classification,
GeoRS(44), No. 8, August 2006, pp. 2305-2318.
IEEE DOI 0608
BibRef

Stathakis, D.[Demetris], Kanellopoulos, I.[Ioannis],
Global Elevation Ancillary Data for Land-use Classification Using Granular Neural Networks,
PhEngRS(74), No. 1, January 2008, pp. 55-64.
WWW Link. 0803
Initial guidelines for the construction of granular neural networks in the remote sensing context. BibRef

Stathakis, D.[Demetris], Kanellopoulos, I.[Ioannis],
Global Optimization versus Deterministic Pruning for the Classification of Remotely Sensed Imagery,
PhEngRS(74), No. 10, October 2008, pp. 1259-1266.
WWW Link. 0804
A method for optimal Multi Layer Perceptron topology determination carried out by a genetic algorithm. BibRef

Arbel, R.[Reuven], Rokach, L.[Lior],
Classifier evaluation under limited resources,
PRL(27), No. 14, 15 October 2006, pp. 1619-1631.
Elsevier DOI 0609
Evaluation measures; Hit-rate; Recall; Receiver operating characteristic BibRef

Nangendo, G.[Grace], Skidmore, A.K.[Andrew K.], van Oosten, H.[Henk],
Mapping East African tropical forests and woodlands: A comparison of classifiers,
PandRS(61), No. 6, February 2007, pp. 393-404.
Elsevier DOI 0703
Forest classification; Conventional classifiers; Expert System; Classification accuracy; East Africa BibRef

An, S.J.[Sen-Jian], Liu, W.Q.[Wan-Quan], Venkatesh, S.[Svetha],
Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression,
PR(40), No. 8, August 2007, pp. 2154-2162.
Elsevier DOI 0704
BibRef
Earlier:
Efficient Cross-validation of the Complete Two Stages in KFD Classifier Formulation,
ICPR06(III: 240-244).
IEEE DOI 0609
Model selection; Cross-validation; Kernel methods BibRef

An, S.[Senjian], Peursum, P.[Patrick], Liu, W.Q.[Wan-Quan], Venkatesh, S.[Svetha],
Efficient subwindow search with submodular score functions,
CVPR11(1409-1416).
IEEE DOI 1106
BibRef
Earlier:
Efficient algorithms for subwindow search in object detection and localization,
CVPR09(264-271).
IEEE DOI 0906
BibRef

An, S.[Senjian], Peursum, P.[Patrick], Liu, W.Q.[Wan-Quan], Venkatesh, S.[Svetha], Chen, X.M.[Xiao-Ming],
Exploiting Monge structures in optimum subwindow search,
CVPR10(926-933).
IEEE DOI 1006
BibRef

Pham, D.S.[Duc-Son], Venkatesh, S.[Svetha],
Robust learning of discriminative projection for multicategory classification on the Stiefel manifold,
CVPR08(1-7).
IEEE DOI 0806
BibRef

Pham, D.S.[Duc-Son], Venkatesh, S.[Svetha],
Joint learning and dictionary construction for pattern recognition,
CVPR08(1-8).
IEEE DOI 0806
BibRef

Bøcher, P.K., McCloy, K.R.,
Optimizing Image Resolution to Maximize the Accuracy of Hard Classification,
PhEngRS(73), No. 8, August 2007, pp. 893-904.
WWW Link. 0709
The relationship between classification accuracy and within class variances is investigated showing that within class variances are a function of image resolution. BibRef

Devarakota, P.R.R.[Pandu Ranga Rao], Mirbach, B.[Bruno], Ottersten, B.[Bjorn],
Reliability estimation of a statistical classifier,
PRL(29), No. 3, 1 February 2008, pp. 243-253.
Elsevier DOI 0801
Pattern classification; Local density estimation; Confidence intervals; Binomial distribution; GMMs; Pattern rejection BibRef

Sahiner, B., Chan, H.P., Hadjiiski, L.M.,
Performance Analysis of Three-Class Classifiers: Properties of a 3-D ROC Surface and the Normalized Volume Under the Surface for the Ideal Observer,
MedImg(27), No. 2, February 2008, pp. 215-227.
IEEE DOI 0802
BibRef

Volkovich, Z., Barzily, Z., Morozensky, L.,
A statistical model of cluster stability,
PR(41), No. 7, July 2008, pp. 2174-2188.
Elsevier DOI 0804
Cluster validation; Negative definite functions; Statistical model BibRef

Akhbardeh, A.[Alireza], Nikhil, Koskinen, P.E.[Perttu E.], Yli-Harja, O.[Olli],
Towards the experimental evaluation of novel supervised fuzzy adaptive resonance theory for pattern classification,
PRL(29), No. 8, 1 June 2008, pp. 1082-1093.
Elsevier DOI 0804
Affine look-up table; Classification; Pre-classification; Post-classification; Supervised fuzzy adaptive resonance theory (SF-ART) network Iris recognition. BibRef

Ferri, C., Hernandez-Orallo, J., Modroiu, R.,
An experimental comparison of performance measures for classification,
PRL(30), No. 1, 1 January 2009, pp. 27-38.
Elsevier DOI 0811
Classification; Performance measures; Ranking; Calibration BibRef

Lago-Fernandez, L.F.[Luis F.], Corbacho, F.[Fernando],
Normality-based validation for crisp clustering,
PR(43), No. 3, March 2010, pp. 782-795.
Elsevier DOI 1001
Crisp clustering; Cluster validation; Negentropy BibRef

Chen, J.[Jin], Zhu, X.L.[Xiao-Lin], Imura, H.[Hidefumi], Chen, X.H.[Xue-Hong],
Consistency of accuracy assessment indices for soft classification: Simulation analysis,
PandRS(65), No. 2, March 2010, pp. 156-164.
Elsevier DOI 1003
Soft classification; Accuracy assessment; Sub-pixel confusion matrix; RMSE; Consistency BibRef

Pascual, D.[Damaris], Pla, F.[Filiberto], Salvador Sánchez, J.,
Cluster validation using information stability measures,
PRL(31), No. 6, 15 April 2010, pp. 454-461.
Elsevier DOI 1004
BibRef
Earlier:
Cluster Stability Assessment Based on Theoretic Information Measures,
CIARP08(219-226).
Springer DOI 0809
Cluster validation; Stability index; Information theory BibRef

Valverde-Albacete, F.J.[Francisco J.], Pelaez-Moreno, C.[Carmen],
Two information-theoretic tools to assess the performance of multi-class classifiers,
PRL(31), No. 12, 1 September 2010, pp. 1665-1671.
Elsevier DOI 1008
Multi-class classifier; Confusion matrix; Contingency table; Performance measure; de Finetti diagram; Entropy triangle BibRef

Lee, Y.R.[Young-Rok], Lee, J.H.[Jeong-Hwa], Jun, C.H.[Chi-Hyuck],
Stability-based validation of bicluster solutions,
PR(44), No. 2, February 2011, pp. 252-264.
Elsevier DOI 1011
Biclustering; Validation; Stability; Resampling BibRef

Fu, X.[Xin], Shen, Q.A.[Qi-Ang],
Fuzzy complex numbers and their application for classifiers performance evaluation,
PR(44), No. 7, July 2011, pp. 1403-1417.
Elsevier DOI 1103
Fuzzy complex numbers; Performance evaluation; Feature selection; Pattern classification BibRef

Wicker, N.[Nicolas],
A note on ball segment picking related to clustering,
PRL(32), No. 5, 1 April 2011, pp. 651-655.
Elsevier DOI 1103
Density of points clustering; DPC; Ball segment picking; Curse of dimensionality; Sampling BibRef

Woloszynski, T.[Tomasz], Kurzynski, M.[Marek],
A probabilistic model of classifier competence for dynamic ensemble selection,
PR(44), No. 10-11, October-November 2011, pp. 2656-2668.
Elsevier DOI 1101
BibRef
Earlier:
A Measure of Competence Based on Randomized Reference Classifier for Dynamic Ensemble Selection,
ICPR10(4194-4197).
IEEE DOI 1008
BibRef
Earlier:
On a New Measure of Classifier Competence Applied to the Design of Multiclassifier Systems,
CIAP09(995-1004).
Springer DOI 0909
Probabilistic modelling; Classifier competence; Multiple classifier system; Beta distribution BibRef

Frasch, J.V.[Janick V.], Lodwich, A.[Aleksander], Shafait, F.[Faisal], Breuel, T.M.[Thomas M.],
A Bayes-true data generator for evaluation of supervised and unsupervised learning methods,
PRL(32), No. 11, 1 August 2011, pp. 1523-1531.
Elsevier DOI 1108
Synthetic data generation; Benchmarking; Experimental proofs BibRef

Chudzian, P.[Pawel],
Evaluation measures for kernel optimization,
PRL(33), No. 9, 1 July 2012, pp. 1108-1116.
Elsevier DOI 1202
Kernel evaluation measures; Kernel optimization; Kernel methods; Radial basis function kernel; Pattern classification Transform pattern to feature space. BibRef

Gey, S.[Servane],
Risk bounds for CART classifiers under a margin condition,
PR(45), No. 9, September 2012, pp. 3523-3534.
Elsevier DOI 1206
Classification; CART; Pruning; Margin; Risk bounds. Classification And Regression Trees (CART) BibRef

Liu, J., Zhang, Z.J., Yang, Y., Wang, M.,
Comments on 'Probabilities of false alarm and detection for the NAMF operating in Gaussian clutter',
SPLetters(19), No. 10, October 2012, pp. 671.
IEEE DOI 1209
BibRef

de França, F.O., Coelho, G.P., von Zuben, F.J.,
Predicting missing values with biclustering: A coherence-based approach,
PR(46), No. 5, May 2013, pp. 1255-1266.
Elsevier DOI 1302
Biclustering; Missing data imputation; Knowledge discovery; Quadratic programming BibRef

Richards, J.A., Kingsbury, N.G.,
Is There a Preferred Classifier for Operational Thematic Mapping?,
GeoRS(52), No. 5, May 2014, pp. 2715-2725.
IEEE DOI 1403
Classification BibRef

Liu, Q.B.[Qing-Bao], Dong, G.Z.[Guo-Zhu],
CPCQ: Contrast pattern based clustering quality index for categorical data,
PR(45), No. 4, 2012, pp. 1739-1748.
Elsevier DOI 1410
Clustering validation BibRef

Wong, T.T.[Tzu-Tsung],
Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation,
PR(48), No. 9, 2015, pp. 2839-2846.
Elsevier DOI 1506
Classification BibRef

Henriques, R.[Rui], Antunes, C.[Cláudia], Madeira, S.C.[Sara C.],
A structured view on pattern mining-based biclustering,
PR(48), No. 12, 2015, pp. 3941-3958.
Elsevier DOI 1509
Biclustering BibRef

Soares, D.F.[Diogo F.], Henriques, R.[Rui], Madeira, S.C.[Sara C.],
Comprehensive assessment of triclustering algorithms for three-way temporal data analysis,
PR(150), 2024, pp. 110303.
Elsevier DOI 2403
Triclustering, Temporal data, Three-way data BibRef

Henriques, R.[Rui], Madeira, S.C.[Sara C.],
FleBiC: Learning classifiers from high-dimensional biomedical data using discriminative biclusters with non-constant patterns,
PR(115), 2021, pp. 107900.
Elsevier DOI 2104
Associative classification, Discriminative paterns, Biclustering, Non-constant patterns, Biomedical data, High-dimensional data BibRef

Park, N.W.[No-Wook], Kyriakidis, P.C.[Phaedon C.], Hong, S.Y.[Suk-Young],
Spatial Estimation of Classification Accuracy Using Indicator Kriging with an Image-Derived Ambiguity Index,
RS(8), No. 4, 2016, pp. 320.
DOI Link 1604
BibRef

Condessa, F.[Filipe], Bioucas-Dias, J.M.[José M.], Kovacevic, J.[Jelena],
Performance measures for classification systems with rejection,
PR(63), No. 1, 2017, pp. 437-450.
Elsevier DOI 1612
Classification with rejection BibRef

Lei, Y.[Yang], Bezdek, J.C.[James C.], Romano, S.[Simone], Vinh, N.X.[Nguyen Xuan], Chan, J.[Jeffrey], Bailey, J.[James],
Ground truth bias in external cluster validity indices,
PR(65), No. 1, 2017, pp. 58-70.
Elsevier DOI 1702
External cluster validity indices BibRef

Wong, T.T.[Tzu-Tsung],
Parametric methods for comparing the performance of two classification algorithms evaluated by k-fold cross validation on multiple data sets,
PR(65), No. 1, 2017, pp. 97-107.
Elsevier DOI 1702
Classification BibRef

Kautz, T.[Thomas], Eskofier, B.M.[Bjoern M.], Pasluosta, C.F.[Cristian F.],
Generic performance measure for multiclass-classifiers,
PR(68), No. 1, 2017, pp. 111-125.
Elsevier DOI 1704
Classification BibRef

Jiang, G.X.[Gao-Xia], Wang, W.J.[Wen-Jian],
Error estimation based on variance analysis of k-fold cross-validation,
PR(69), No. 1, 2017, pp. 94-106.
Elsevier DOI 1706
Error estimation BibRef

Gossmann, A.[Alexej], Zille, P.[Pascal], Calhoun, V.[Vince], Wang, Y.P.[Yu-Ping],
FDR-Corrected Sparse Canonical Correlation Analysis With Applications to Imaging Genomics,
MedImg(37), No. 8, August 2018, pp. 1761-1774.
IEEE DOI 1808
False Discovery Rate. Genomics, Bioinformatics, Correlation, Brain, Functional magnetic resonance imaging, Testing, fMRI analysis, probabilistic and statistical methods BibRef

Boonprong, S.[Sornkitja], Cao, C.X.[Chun-Xiang], Chen, W.[Wei], Ni, X.L.[Xi-Liang], Xu, M.[Min], Acharya, B.K.[Bipin Kumar],
The Classification of Noise-Afflicted Remotely Sensed Data Using Three Machine-Learning Techniques: Effect of Different Levels and Types of Noise on Accuracy,
IJGI(7), No. 7, 2018, pp. xx-yy.
DOI Link 1808
BibRef

Schlüter, R.[Ralf], Beck, E.[Eugen], Ney, H.[Hermann],
Upper and Lower Tight Error Bounds for Feature Omission with an Extension to Context Reduction,
PAMI(41), No. 2, February 2019, pp. 502-514.
IEEE DOI 1901
Context modeling, Analytical models, Upper bound, Feature extraction, Measurement uncertainty, sequence classification BibRef

Wang, Y.Q.[Yong-Qiao], Li, L.S.[Li-Shuai], Dang, C.Y.[Chuang-Yin],
Calibrating Classification Probabilities with Shape-Restricted Polynomial Regression,
PAMI(41), No. 8, August 2019, pp. 1813-1827.
IEEE DOI 1907
Calibration, Computational modeling, Support vector machines, Estimation, Symmetric matrices, Training data, polynomial regression BibRef

Zhang, Q.[Qi], Zhang, P.L.[Peng-Lin],
An Uncertainty Descriptor for Quantitative Measurement of the Uncertainty of Remote Sensing Images,
RS(11), No. 13, 2019, pp. xx-yy.
DOI Link 1907
Reliability of the classification. BibRef

Gweon, H.[Hyukjun], Yu, H.[Hao],
How reliable is your reliability diagram?,
PRL(125), 2019, pp. 687-693.
Elsevier DOI 1909
Probabilistic classifier evaluation, Statistical testing, Reliability diagram, Poisson binomial BibRef

Hanczar, B.[Blaise],
Performance visualization spaces for classification with rejection option,
PR(96), 2019, pp. 106984.
Elsevier DOI 1909
Classification with reject option, Classifier performances BibRef

Radoux, J.[Julien], Waldner, F.[François], Bogaert, P.[Patrick],
How Response Designs and Class Proportions Affect the Accuracy of Validation Data,
RS(12), No. 2, 2020, pp. xx-yy.
DOI Link 2001
BibRef

Elmes, A.[Arthur], Alemohammad, H.[Hamed], Avery, R.[Ryan], Caylor, K.[Kelly], Eastman, J.R.[J. Ronald], Fishgold, L.[Lewis], Friedl, M.A.[Mark A.], Jain, M.[Meha], Kohli, D.[Divyani], Bayas, J.C.L.[Juan Carlos Laso], Lunga, D.[Dalton], McCarty, J.L.[Jessica L.], Pontius, R.G.[Robert Gilmore], Reinmann, A.B.[Andrew B.], Rogan, J.[John], Song, L.[Lei], Stoynova, H.[Hristiana], Ye, S.[Su], Yi, Z.F.[Zhuang-Fang], Estes, L.[Lyndon],
Accounting for Training Data Error in Machine Learning Applied to Earth Observations,
RS(12), No. 6, 2020, pp. xx-yy.
DOI Link 2003
BibRef

Soleymani, R.[Roghayeh], Granger, E.[Eric], Fumera, G.[Giorgio],
F-measure curves: A tool to visualize classifier performance under imbalance,
PR(100), 2020, pp. 107146.
Elsevier DOI 2005
Pattern classification, Class imbalance, Performance metrics, F-measure, Visualization tools, Video face recognition BibRef

Jeong, M., Dytso, A., Cardone, M.,
Gradient of Error Probability of M-ary Hypothesis Testing Problems Under Multivariate Gaussian Noise,
SPLetters(27), 2020, pp. 1909-1913.
IEEE DOI 2011
Error probability, Testing, Covariance matrices, Gaussian noise, Random variables, Noise measurement, Matrices, Error probability, multivariate Gaussian noise BibRef

Radoux, J.[Julien], Bogaert, P.[Patrick],
About the Pitfall of Erroneous Validation Data in the Estimation of Confusion Matrices,
RS(12), No. 24, 2020, pp. xx-yy.
DOI Link 2012
BibRef

Wang, P.[Pin], Fan, E.[En], Wang, P.[Peng],
Comparative analysis of image classification algorithms based on traditional machine learning and deep learning,
PRL(141), 2021, pp. 61-67.
Elsevier DOI 2101
Traditional machine learning, Deep learning, Support vector machines, Convolutional neural networks BibRef

Tan, Q.L.[Qu-Lin], Guo, B.[Bin], Hu, J.[Jun], Dong, X.F.[Xiao-Feng], Hu, J.P.[Ji-Ping],
Object-oriented remote sensing image information extraction method based on multi-classifier combination and deep learning algorithm,
PRL(141), 2021, pp. 32-36.
Elsevier DOI 2101
Multi-classifier combination, Deep learning algorithm, Object-oriented, Remote sensing image, Information extraction BibRef

Espinosa, S.[Sebastian], Silva, J.F.[Jorge F.], Piantanida, P.[Pablo],
Finite-Length Bounds on Hypothesis Testing Subject to Vanishing Type I Error Restrictions,
SPLetters(28), 2021, pp. 229-233.
IEEE DOI 2102
Binary Hypothesis Testing (BHT). Convergence, Error probability, Cascading style sheets, Upper bound, Task analysis, Bayes methods, Tools, Error exponent, concentration inequalities BibRef

Yousef, W.A.[Waleed A.],
Estimating the standard error of cross-Validation-Based estimators of classifier performance,
PRL(146), 2021, pp. 115-125.
Elsevier DOI 2105
Cross validation, CV, Uncertainty, Variance, Influence function, Influence curve, Components of variance, Classification BibRef

Tavakkol, B.[Behnam], Choi, J.[Jeongsub], Jeong, M.K.[Myong Kee], Albin, S.L.[Susan L.],
Object-based cluster validation with densities,
PR(121), 2022, pp. 108223.
Elsevier DOI 2109
Clustering, Clustering validity index, Internal index, Density-based cluster validation, Unsupervised BibRef

Lee, J.[Jaemin], Han, M.[Minseok], Lee, J.S.[Jong-Seok],
Convergence analysis of connection center evolution and faster clustering,
PR(127), 2022, pp. 108639.
Elsevier DOI 2205
Clustering, Center evolution, Convergence analysis, Ergodic Markov chain, Faster algorithm BibRef

Tank, A.[Alex], Covert, I.[Ian], Foti, N.[Nicholas], Shojaie, A.[Ali], Fox, E.B.[Emily B.],
Neural Granger Causality,
PAMI(44), No. 8, August 2022, pp. 4267-4279.
IEEE DOI 2207
Error analysis of reduced model and full model. Time series analysis, Neural networks, Reactive power, Recurrent neural networks, Predictive models, Estimation, interpretability BibRef

Li, C.Y.[Cong-Yu], Li, Z.[Zhen], Liu, X.X.[Xin-Xin], Li, S.T.[Shu-Tao],
The Influence of Image Degradation on Hyperspectral Image Classification,
RS(14), No. 20, 2022, pp. xx-yy.
DOI Link 2211
BibRef

Foody, G.M.[Giles M.],
Global and Local Assessment of Image Classification Quality on an Overall and Per-Class Basis without Ground Reference Data,
RS(14), No. 21, 2022, pp. xx-yy.
DOI Link 2212
BibRef

Ali, M.[Mehdi], Berrendorf, M.[Max], Hoyt, C.T.[Charles Tapley], Vermue, L.[Laurent], Galkin, M.[Mikhail], Sharifzadeh, S.[Sahand], Fischer, A.[Asja], Tresp, V.[Volker], Lehmann, J.[Jens],
Bringing Light Into the Dark: A Large-Scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework,
PAMI(44), No. 12, December 2022, pp. 8825-8845.
IEEE DOI 2212
All code and tests:
WWW Link. and
WWW Link. Evaluation of 21 models with benchmark testing. Computational modeling, Benchmark testing, Training, Predictive models, Task analysis, benchmarking BibRef

Maldonado, S.[Sebastián], Saltos, R.[Ramiro], Vairetti, C.[Carla], Delpiano, J.[José],
Mitigating the effect of dataset shift in clustering,
PR(134), 2023, pp. 109058.
Elsevier DOI 2212
Induced ordered weighted average, Kernel k-means, OWA operators, Dataset shift, Clustering BibRef

Yamaguchi, T.[Takumi], Murakawa, M.[Masahiro],
Mixup gamblers+: Learning interpolated pseudo 'uncertainty' in latent feature space for reliable inference,
PRL(164), 2022, pp. 191-199.
Elsevier DOI 2212
Reliability of inference results. Selective classification, Classification with a reject option, Metric learning, Mixup, Uncertainty estimation, Confidence calibration BibRef

Ahmadzadeh, A.[Azim], Kempton, D.J.[Dustin J.], Martens, P.C.[Petrus C.], Angryk, R.A.[Rafal A.],
Contingency Space: A Semimetric Space for Classification Evaluation,
PAMI(45), No. 2, February 2023, pp. 1501-1513.
IEEE DOI 2301
Measurement, Extraterrestrial measurements, Performance evaluation, Analytical models, Indexes, Sensitivity, knowledge representation formalisms and methods BibRef

Wang, J.T.[Jie-Ting], Qian, Y.H.[Yu-Hua], Li, F.J.[Fei-Jiang], Liang, J.[Jiye], Zhang, Q.F.[Qing-Fu],
Generalization Performance of Pure Accuracy and its Application in Selective Ensemble Learning,
PAMI(45), No. 2, February 2023, pp. 1798-1816.
IEEE DOI 2301
Loss measurement, Particle measurements, Atmospheric measurements, Support vector machines, selective ensemble learning BibRef

Fränti, P.[Pasi], Mariescu-Istodor, R.[Radu],
Soft precision and recall,
PRL(167), 2023, pp. 115-121.
Elsevier DOI 2303
Soft measures, Evaluation, Precision, Recall, F-score BibRef

Masana, M.[Marc], Liu, X.[Xialei], Twardowski, B.[Bartlomiej], Menta, M.[Mikel], Bagdanov, A.D.[Andrew D.], van de Weijer, J.[Joost],
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification,
PAMI(45), No. 5, May 2023, pp. 5513-5533.
IEEE DOI 2304
Task analysis, Training, Network architecture, Learning systems, Image classification, Training data, Privacy, catastrophic forgetting BibRef

Viering, T.[Tom], Loog, M.[Marco],
The Shape of Learning Curves: A Review,
PAMI(45), No. 6, June 2023, pp. 7799-7819.
IEEE DOI 2305
Training, Shape, Behavioral sciences, Training data, Loss measurement, Computational modeling, Standards, regression BibRef

Chen, Q.Q.[Qing-Qiang], Cao, F.Y.[Fu-Yuan], Xing, Y.[Ying], Liang, J.[Jiye],
Evaluating Classification Model Against Bayes Error Rate,
PAMI(45), No. 8, August 2023, pp. 9639-9653.
IEEE DOI 2307
Task analysis, Noise measurement, Estimation, Error analysis, Reliability theory, Data models, Support vector machines, percolation theory BibRef

Zhou, J.K.[Jing-Kai], Wang, P.C.[Pi-Chao], Tang, J.S.[Jia-Sheng], Wang, F.[Fan], Liu, Q.[Qiong], Li, H.[Hao], Jin, R.[Rong],
What Limits the Performance of Local Self-attention?,
IJCV(131), No. 10, October 2023, pp. 2516-2528.
Springer DOI 2309
BibRef

Qu, H.X.[Hao-Xuan], Foo, L.G.[Lin Geng], Li, Y.C.[Yan-Chao], Liu, J.[Jun],
Towards More Reliable Confidence Estimation,
PAMI(45), No. 11, November 2023, pp. 13152-13169.
IEEE DOI 2310
BibRef

Liu, S.X.[Shi-Xia],
Enhancing Training Data Quality With Visual Analytics,
Computer(56), No. 11, November 2023, pp. 4-6.
IEEE DOI 2310
BibRef

Campello, B.S.C.[Betania Silva Carneiro], Duarte, L.T.[Leonardo Tomazeli], Romano, J.M.T.[João Marcos Travassos],
Multicriteria decision support employing adaptive prediction in a tensor-based feature representation,
PRL(174), 2023, pp. 52-56.
Elsevier DOI 2310
Adaptive prediction methods, Multi-criteria decision analysis, MCDA, Temporal analysis, Multi-period, Dynamic multi-attribute decision making BibRef

Zhang, X.[Xiao], Wei, W.[Wei], Zhang, Z.[Zhen], Zhang, L.[Lei], Li, W.[Wei],
Milstein-driven neural stochastic differential equation model with uncertainty estimates,
PRL(174), 2023, pp. 71-77.
Elsevier DOI 2310
Uncertainty quantification, Stochastic differential equation, Milstein-driven SDE-net BibRef

Huang, Y.[Yan], Zhang, Z.[Zhang], Huang, Y.[Yan], Wu, Q.[Qiang], Huang, H.[Han], Zhong, Y.[Yi], Wang, L.[Liang],
Customized meta-dataset for automatic classifier accuracy evaluation,
PR(146), 2024, pp. 110026.
Elsevier DOI 2311
Auto-evaluation, Automatic classifier accuracy evaluation BibRef

Montalvão, J.[Jugurta], Duarte, D.[Dami], Boccato, L.[Levy],
A coincidence detection perspective for the maximum mean discrepancy,
PRL(177), 2024, pp. 20-25.
Elsevier DOI 2401
Aids in interpretability. Coincidence detection, Maximum mean discrepancy, Collision entropy, Hypothesis test, Grassberger-Procaccia method BibRef

Liu, Z.W.[Zi-Wei], Miao, Z.Q.[Zhong-Qi], Zhan, X.H.[Xiao-Hang], Wang, J.Y.[Jia-Yun], Gong, B.Q.[Bo-Qing], Yu, S.X.[Stella X.],
Open Long-Tailed Recognition in a Dynamic World,
PAMI(46), No. 3, March 2024, pp. 1836-1851.
IEEE DOI 2402
BibRef
Earlier:
Large-Scale Long-Tailed Recognition in an Open World,
CVPR19(2532-2541).
IEEE DOI 2002
Tail, Visualization, Head, Training, Task analysis, Measurement, Long-tailed recognition, few-shot learning, active learning BibRef

Alon, I.[Itai], Arnon, D.[David], Wiesel, A.[Ami],
Learning Minimal Volume Uncertainty Ellipsoids,
SPLetters(31), 2024, pp. 1655-1659.
IEEE DOI 2407
Ellipsoids, Uncertainty, Training, Shape, Calibration, Estimation, Covariance matrices, Uncertainty ellipsoid, conformal prediction BibRef

Tiotsop, L.F.[Lohic Fotio], Servetti, A.[Antonio], Barkowsky, M.[Marcus], Masala, E.[Enrico],
Modeling Subject Scoring Behaviors in Subjective Experiments Based on a Discrete Quality Scale,
MultMed(26), 2024, pp. 8742-8757.
IEEE DOI 2408
Noise measurement, Probabilistic logic, Numerical models, Media, Systematics, Recommender systems, Proposals, subject inconsistency BibRef

Minatel, D.[Diego], Parmezan, A.R.S.[Antonio R. S.], Cúri, M.[Mariana], de Andrade Lopes, A.[Alneu],
DIF-SR: A Differential Item Functioning-based Sample Reweighting Method,
CIARP23(I:630-645).
Springer DOI 2312
identify bias. BibRef

Mukhoti, J.[Jishnu], Kirsch, A.[Andreas], van Amersfoort, J.[Joost], Torr, P.H.S.[Philip H.S.], Gal, Y.[Yarin],
Deep Deterministic Uncertainty: A New Simple Baseline,
CVPR23(24384-24394)
IEEE DOI 2309
BibRef

Jiang, Q.[Qian], Chen, C.Y.[Chang-You], Zhao, H.[Han], Chen, L.Q.[Li-Qun], Ping, Q.[Qing], Tran, S.D.[Son Dinh], Xu, Y.[Yi], Zeng, B.[Belinda], Chilimbi, T.[Trishul],
Understanding and Constructing Latent Modality Structures in Multi-Modal Representation Learning,
CVPR23(7661-7671)
IEEE DOI 2309
BibRef

Wang, D.B.[Deng-Bao], Li, L.Q.[Lan-Qing], Zhao, P.L.[Pei-Lin], Heng, P.A.[Pheng-Ann], Zhang, M.L.[Min-Ling],
On the Pitfall of Mixup for Uncertainty Calibration,
CVPR23(7609-7618)
IEEE DOI 2309
BibRef

Agarwal, A.[Akshay], Ratha, N.[Nalini], Vatsa, M.[Mayank], Singh, R.[Richa],
Benchmarking Robustness Beyond LP Norm Adversaries,
AdvRob22(342-359).
Springer DOI 2304
BibRef

Risser-Maroix, O.[Olivier], Chamand, B.[Benjamin],
What can we Learn by Predicting Accuracy?,
WACV23(2389-2398)
IEEE DOI 2302
Correlation, Pipelines, Machine learning, Feature extraction, Linear programming, Task analysis, ethical computer vision BibRef

Ueda, R.[Ryosuke], Takeuchi, K.[Koh], Kashima, H.[Hisashi],
Mitigating Observatio> ICPR22,
(1171-1177)
IEEE DOI 2212
Crowdsourcing, Costs, Correlation, Supervised learning, Robustness, Complexity theory BibRef

Pliushch, I.[Iuliia], Mundt, M.[Martin], Lupp, N.[Nicolas], Ramesh, V.[Visvanathan],
When Deep Classifiers Agree: Analyzing Correlations Between Learning Order and Image Statistics,
ECCV22(VIII:397-413).
Springer DOI 2211
BibRef

Zhu, F.[Fei], Cheng, Z.[Zhen], Zhang, X.Y.[Xu-Yao], Liu, C.L.[Cheng-Lin],
Rethinking Confidence Calibration for Failure Prediction,
ECCV22(XXV:518-536).
Springer DOI 2211

WWW Link. Confidence. BibRef

Moayeri, M.[Mazda], Pope, P.[Phillip], Balaji, Y.[Yogesh], Feizi, S.[Soheil],
A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes,
CVPR22(19065-19075)
IEEE DOI 2210
Training, Location awareness, Adaptation models, Visualization, Sensitivity, Annotations, Datasets and evaluation, Visual reasoning BibRef

Poms, F.[Fait], Sarukkai, V.[Vishnu], Mullapudi, R.T.[Ravi Teja], Sohoni, N.S.[Nimit S.], Mark, W.R.[William R.], Ramanan, D.[Deva], Fatahalian, K.[Kayvon],
Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories,
ICCV21(10685-10694)
IEEE DOI 2203
Monte Carlo methods, Machine learning algorithms, Costs, Computational modeling, Training data, Machine learning, Transfer/Low-shot/Semi/Unsupervised Learning BibRef

Moraes, D., Benevides, P., Moreira, F.D., Costa, H., Caetano, M.,
Exploring the Use of Classification Uncertainty to Improve Classification Accuracy,
ISPRS21(B3-2021: 81-86).
DOI Link 2201
BibRef

Wang, C.Y.[Chien-Yao], Liao, H.Y.M.[Hong-Yuan Mark], Yeh, I.H.[I-Hau], Chuang, Y.Y.[Yung-Yu], Lin, Y.L.[Youn-Long],
Exploring the power of lightweight YOLOv4,
LPCV21(779-788)
IEEE DOI 2112
Training, Learning systems, Power demand, Computational modeling, Neural networks, Pipelines, Object detection BibRef

Deng, W.J.[Wei-Jian], Zheng, L.[Liang],
AutoEval: Are Labels Always Necessary for Classifier Accuracy Evaluation?,
PAMI(46), No. 3, March 2024, pp. 1868-1880.
IEEE DOI 2402
BibRef
Earlier:
Are Labels Always Necessary for Classifier Accuracy Evaluation?,
CVPR21(15064-15073)
IEEE DOI 2111
Training, Correlation, Task analysis, Predictive models, Standards, Neural networks, Image color analysis, dataset-level regression. Computational modeling, Rendering (computer graphics), Object recognition BibRef

Sensoy, M.[Murat], Saleki, M.[Maryam], Julier, S.[Simon], Aydogan, R.[Reyhan], Reid, J.[John],
Misclassification Risk and Uncertainty Quantification in Deep Classifiers,
WACV21(2483-2491)
IEEE DOI 2106
Training, Deep learning, Uncertainty, Decision making, Predictive models BibRef

Shen, Y.C.[Yi-Chen], Zhang, Z.L.[Zhi-Lu], Sabuncu, M.R.[Mert R.], Sun, L.[Lin],
Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillation,
WACV21(707-716)
IEEE DOI 2106
Deep learning, Training, Uncertainty, Computational modeling, Semantics, Estimation BibRef

Serrat, J.[Joan], Ruiz, I.[Idoia],
Rank-based ordinal classification,
ICPR21(8069-8076)
IEEE DOI 2105
Some errors are worse than others. Measurement, Satellites, Buildings, Estimation, Network architecture, Benchmark testing BibRef

Burghouts, G.J.[Gertjan J.],
Task-specific Novel Object Characterization,
HCAU20(447-455).
Springer DOI 2103
Deal with unknown objects. BibRef

Gao, J.[Jian], Hua, Y.[Yang], Hu, G.S.[Guo-Sheng], Wang, C.[Chi], Robertson, N.M.[Neil M.],
Reducing Distributional Uncertainty by Mutual Information Maximisation and Transferable Feature Learning,
ECCV20(XXIII:587-605).
Springer DOI 2011
Effect of distributions variations. BibRef

Philion, J.[Jonah], Kar, A.[Amlan], Fidler, S.[Sanja],
Learning to Evaluate Perception Models Using Planner-Centric Metrics,
CVPR20(14052-14061)
IEEE DOI 2008
Code, Evaluation.
WWW Link. Detectors, Task analysis, Object detection, Noise measurement, Trajectory BibRef

Wang, P.[Pei], Vasconcelos, N.M.[Nuno M.],
SCOUT: Self-Aware Discriminant Counterfactual Explanations,
CVPR20(8978-8987)
IEEE DOI 2008
Visualization, Heat Maps, Task analysis, Training, Protocols, BibRef

Simon, D., Farber, M., Goldenberg, R.,
Auto-Annotation Quality Prediction for Semi-Supervised Learning with Ensembles,
VL3W20(3984-3988)
IEEE DOI 2008
Training, Data models, Semantics, Manuals, Predictive models, Task analysis, Labeling BibRef

Branchaud-Charron, F.[Frederic], Achkar, A.[Andrew], Jodoin, P.M.[Pierre-Marc],
Spectral Metric for Dataset Complexity Assessment,
CVPR19(3210-3219).
IEEE DOI 2002
BibRef

Das, D.[Dipanjan], Ghosh, R.[Ratul], Bhowmick, B.[Brojeshwar],
Deep Representation Learning Characterized by Inter-Class Separation for Image Clustering,
WACV19(628-637)
IEEE DOI 1904
feature extraction, image representation, learning (artificial intelligence), neural nets, Deep learning BibRef

Chen, Z.Q.[Zhi-Qiang], Du, C.D.[Chang-De], Huang, L.J.[Li-Jie], Li, D.[Dan], He, H.G.[Hui-Guang],
Improving Image Classification Performance with Automatically Hierarchical Label Clustering,
ICPR18(1863-1868)
IEEE DOI 1812
Task analysis, Training, Merging, Dogs, Labeling, Machine learning, Clustering algorithms BibRef

Fawzi, A.[Alhussein], Frossard, P.[Pascal],
Measuring the effect of nuisance variables on classifiers,
BMVC16(xx-yy).
HTML Version. 1805
BibRef

Mendiola-Lau, V.[Victor], Silva Mata, F.J.[Francisco José], Calaña, Y.P.[Yenisel Plasencia], Bustamante, I.T.[Isneri Talavera], de Marsico, M.[Maria],
Bio-Chemical Data Classification by Dissimilarity Representation and Template Selection,
CIARP17(374-381).
Springer DOI 1802
BibRef

Barddal, J.P., Gomes, H.M., de Souza Britto, A., Enembreck, F.,
A benchmark of classifiers on feature drifting data streams,
ICPR16(2180-2185)
IEEE DOI 1705
Adaptation models, Benchmark testing, Context, Decision trees, Detectors, Feature extraction, Generators BibRef

Wang, D.[Dong], Tan, X.Y.[Xiao-Yang],
Label-Denoising Auto-encoder for Classification with Inaccurate Supervision Information,
ICPR14(3648-3653)
IEEE DOI 1412
Data models BibRef

Cabrera, G.F.[Guillermo F.], Miller, C.J.[Christopher J.], Schneider, J.[Jeff],
Systematic Labeling Bias: De-biasing Where Everyone is Wrong,
ICPR14(4417-4422)
IEEE DOI 1412
Accuracy; Gold; Labeling; Logistics; Spirals; Standards; Training BibRef

Hajizadeh, S.[Siamak], Li, Z.L.[Zi-Li], Dollevoet, R.P.B.J.[Rolf P.B.J.], Tax, D.M.J.[David M.J.],
Evaluating Classification Performance with only Positive and Unlabeled Samples,
SSSPR14(233-242).
Springer DOI 1408
BibRef

Santos, J.M.[Jorge M.], Embrechts, M.[Mark],
A Family of Two-Dimensional Benchmark Data Sets and Its Application to Comparing Different Cluster Validation Indices,
MCPR14(41-50).
Springer DOI 1407
BibRef

Aghazadeh, O.[Omid], Carlsson, S.[Stefan],
Properties of Datasets Predict the Performance of Classifiers,
BMVC13(xx-yy).
DOI Link 1402
BibRef

Loyola-González, O.[Octavio], Martínez-Trinidad, J.F.[José Francisco], Carrasco-Ochoa, J.A.[Jesús Ariel], García-Borroto, M.[Milton],
A Novel Contrast Pattern Selection Method for Class Imbalance Problems,
MCPR17(42-52).
Springer DOI 1706
BibRef
Earlier:
Correlation of Resampling Methods for Contrast Pattern Based Classifiers,
MCPR15(93-102).
Springer DOI 1506
BibRef

García-Borroto, M.[Milton], Loyola-Gonzalez, O.[Octavio], Martínez-Trinidad, J.F.[José Francisco],
Comparing Quality Measures for Contrast Pattern Classifiers,
CIARP13(I:311-318).
Springer DOI 1311
BibRef

Welinder, P.[Peter], Welling, M.[Max], Perona, P.[Pietro],
A Lazy Man's Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration,
CVPR13(3262-3269)
IEEE DOI 1309
BibRef

Wei, S.M.[Si-Ming], Yu, Y.Z.[Yi-Zhou],
Subspace segmentation with a Minimal Squared Frobenius Norm Representation,
ICPR12(3509-3512).
WWW Link. 1302
performs data clustering by solving a convex optimization problem BibRef

Berrar, D.[Daniel],
Null QQ plots: A simple graphical alternative to significance testing for the comparison of classifiers,
ICPR12(1852-1855).
WWW Link. 1302
BibRef

Li, J.[James], Sonmez, A.[Abdullah], Cataltepe, Z.[Zehra], Bax, E.[Eric],
Validation of Network Classifiers,
SSSPR12(448-457).
Springer DOI 1211
probably approximately correct bounds for classifier BibRef

Kallel, M.[Manel], Naouai, M.[Mohamed], Slama, Y.[Yosr],
New Metrics to Evaluate Pattern Recognition in Remote Sensing Images,
CIARP12(664-673).
Springer DOI 1209
BibRef

Ryazanov, V.V.[Vladimir V.],
Estimations of Clustering Quality via Evaluation of Its Stability,
CIARP14(432-439).
Springer DOI 1411
BibRef
Earlier:
Clustering of Incomplete Data and Evaluation of Clustering Quality,
CIARP12(146-153).
Springer DOI 1209
BibRef

Zhou, G.J.[Gui-Jun], Wu, B.[Bo], Li, M.M.[Meng-Meng],
Improved accuracy assessment indices for object-based high resolution remotely sensed imagery classification,
IASP11(181-186).
IEEE DOI 1112
Evaluation using regions, not just pixels. BibRef

Marsetic, A., Kokalj, Z., Ostir, K.,
The effect of lossy image compression on object based image classification: WorldView-2 case study,
HighRes11(xx-yy).
PDF File. 1106
BibRef

Weih, Jr., R.C.[Robert C.], Riggan, Jr., N.D.[Norman D.],
Object-Based Classification vs. Pixel-Based Classification: Comparitive Importance of Multi-Resolution Imagery,
GEOBIA10(xx-yy).
PDF File. 1007
BibRef

Huang, H.Q.[Hai-Qiao], Mok, P.Y.[Pik-Yin], Kwok, Y.L.[Yi-Lin], Au, S.C.[Sau-Chuen],
A Parameter Free Approach for Clustering Analysis,
CAIP09(816-823).
Springer DOI 0909
BibRef

Zhang, W.[Wei], Deng, H.L.[Hong-Li],
Understanding visual dictionaries via Maximum Mutual Information curves,
ICPR08(1-4).
IEEE DOI 0812
BibRef

Zhang, X.[Xiao], Liang, L.[Lin], Tang, X.[Xiaoou], Shum, H.Y.[Heung-Yeung],
L1 regularized projection pursuit for additive model learning,
CVPR08(1-8).
IEEE DOI 0806
BibRef

Su, Y.[Yu], Shan, S.G.[Shi-Guang], Chen, X.L.[Xi-Lin], Gao, W.[Wen],
Classifiability-based Optimal Discriminatory Projection Pursuit,
CVPR08(1-7).
IEEE DOI 0806
BibRef

González-Guevara, V.I.[Víctor Iván], Godoy-Calderon, S.[Salvador], Alba-Cabrera, E.[Eduardo], Ibarra-Fiallo, J.[Julio],
A Mixed Learning Strategy for Finding Typical Testors in Large Datasets,
CIARP15(716-723).
Springer DOI 1511
BibRef

Alba-Cabrera, E.[Eduardo], Ibarra-Fiallo, J.[Julio], Godoy-Calderon, S.[Salvador], Cervantes-Alonso, F.[Fernando],
YYC: A Fast Performance Incremental Algorithm for Finding Typical Testors,
CIARP14(416-423).
Springer DOI 1411
BibRef
Earlier: A1, A2, A3, Only:
A Theoretical and Practical Framework for Assessing the Computational Behavior of Typical Testor-Finding Algorithms,
CIARP13(I:351-358).
Springer DOI 1311
BibRef

Godoy-Calderón, S.[Salvador], Martínez-Trinidad, J.F.[José Francisco], Cortés, M.L.[Manuel Lazo],
Proposal for a Unified Methodology for Evaluating Supervised and Non-supervised Classification Algorithms,
CIARP06(674-685).
Springer DOI 0611
BibRef

Mouchere, H.[Harold], Anquetil, E.[Eric],
A Unified Strategy to Deal with Different Natures of Reject,
ICPR06(II: 792-795).
IEEE DOI 0609
BibRef

Pugliese, L.[Luca], Scarpetta, S.[Silvia], Esposito, A.[Anna], Marinaro, M.[Maria],
An Application of Neural and Probabilistic Unsupervised Methods to Environmental Factor Analysis of Multi-spectral Images,
CIAP05(1190-1197).
Springer DOI 0509
compare 2 clustering methods for LANDSAT classification. BibRef

Maillard, P.[Philippe], Clausi, D.A.[David A.],
Comparing Classification Metrics for Labeling Segmented Remote Sensing Images,
CRV05(421-428).
IEEE DOI 0505
BibRef

Kolsch, T., Keysers, D., Ney, H., Paredes, R.,
Enhancements for local feature based image classification,
ICPR04(I: 248-251).
IEEE DOI 0409
Decompose and analyze nearest neighbor search and direct voting for classification. BibRef

Mansilla, E.B., Ho, T.K.[Tin Kam],
On classifier domains of competence,
ICPR04(I: 136-139).
IEEE DOI 0409
Where are which classifiers competent. BibRef

Johnson, A.Y.[Amos Y.], Sun, J.[Jie], Bobick, A.F.[Aaron F.],
Predicting Large Population Data Cumulative Match Characteristic Performance from Small Population Data,
AVBPA03(821-829).
Springer DOI 0310
BibRef

Lucas, S.M.,
Web-based evaluation and deployment of pattern recognizers,
ICPR02(III: 419-422).
IEEE DOI 0211
BibRef

Duin, R.P.W., Pekalska, E., Tax, D.M.J.,
The characterization of classification problems by classifier disagreements,
ICPR04(I: 140-143).
IEEE DOI 0409
BibRef

de Ridder, D., Pekalska, E., Duin, R.P.W.,
The economics of classification: error vs. complexity,
ICPR02(II: 244-247).
IEEE DOI 0211
BibRef

Varma, M., Zisserman, A.P.,
Classifying materials from images: to cluster or not to cluster?,
Texture02(139-144). 0207
BibRef

Yang, M.H., Roth, D., Ahuja, N.,
A Tale of Two Classifiers: SNoW vs. SVM in Visual Recognition,
ECCV02(IV: 685 ff.).
Springer DOI 0205
BibRef

Huijsmans, N.P., Sebe, N.,
Extended Performance Graphs for Cluster Retrieval,
CVPR01(I:26-31).
IEEE DOI 0110
In classification, issue of how to measure the performance expecially when true negatives are dominant. BibRef

Bromiley, P.A., Courtney, P., Thacker, N.A.,
A Case Study in the use of ROC curves for Algorithm Design,
BMVC01(Poster Session 1).
HTML Version. University of Manchester 0110
BibRef

Michaels, R., Boult, T.E.,
A Stratified Methodology for Classifier and Recognizer Evaluation,
EEMCV01(xx-yy). 0110
BibRef

Tang, M., Xiao, J., Ma, S.D.[Song De],
Two-step Classification Based on Scale Space,
ICPR00(Vol II: 899-902).
IEEE DOI 0009
BibRef

Scott, M.J.J., Niranjan, M., Prager, R.W.,
Realisable Classifiers: Improving Operating Performance on Variable Cost Problems,
BMVC98(xx-yy). BibRef 9800

Prager, R.W.,
CART/CMAC hybrid: regression trees with interpolation,
ICPR94(B:476-478).
IEEE DOI 9410
BibRef

Kraaijveld, M.A.[Martin A.],
An Experimental Comparison of Non-Parametric Classifiers for Time-Constrained Classification Tasks,
ICPR98(Vol I: 428-435).
IEEE DOI 9808
BibRef

Lashkia, V.[Vakhtang], Aleshin, S.,
Test Feature Classifiers: Performance and Applications,
ICPR98(Vol I: 341-343).
IEEE DOI 9808
Properties BibRef

Zhao, W.Y.[Wen-Yi], Chellappa, R.[Rama], Nandhakumar, N.,
Empirical Performance Analysis of Linear Discriminant Classifiers,
CVPR98(164-169).
IEEE DOI BibRef 9800

Gorski, N.,
Optimizing Error-Reject Trade-Off in Recognition Systems,
ICDAR97(1092-1096).
IEEE DOI 9708
BibRef

Altamirano-Robles, L., Eckstein, W.,
The Importance of Feature Visibility for the Evaluation of a Matching Hypothesis,
ICPR96(I: 585-589).
IEEE DOI 9608
(Technical Univ. of Munich, D) BibRef

Yu, S.[Shan], Weigl, K.[Konrad],
Performance comparison of a deterministic and a stochastic method for image classification,
CAIP95(826-831).
Springer DOI 9509
BibRef

Ford, S.J., Hampshire, II, J.B., McKeown, D.M.,
Performance Evaluation of Multispectral Analysis for Surface Material Classification,
DARPA93(421-435). Classification. BibRef 9300

Niblack, W., Petkovic, D., Damian, D.,
Experiments and Evaluations of Rule Based Methods in Image Analysis,
CVPR88(123-128).
IEEE DOI BibRef 8800

Chapter on Pattern Recognition, Clustering, Statistics, Grammars, Learning, Neural Nets, Genetic Algorithms continues in
Error Estimation, Classification Accuracy .