16.7.1.2 Evaluation, Surveillance Systems

Chapter Contents (Back)
Evaluation, Surveillance.

LHI Surveillance Dataset,
Annotated surveillance images. Online2008
HTML Version. Dataset, Segmentation. Subset of larger dataset.
See also Lotus Hill Institute. BibRef 0800

CLEAR: Classification of Events, Activities and Relationships,
MTPH07
WWW Link. Dataset, Activity Recogniton. BibRef 0700

i-LIDS: Bag and vehicle detection challenge,
Online2007 BibRef 0700 AVSBS07
HTML Version. Dataset, Activity Recogniton. Data used at Advanced Video and Signal Based Surveillance, 2007. BibRef

Multimedia Event Detection,
Series of Event and Activity Detection evaluations.
WWW Link.
WWW Link.
WWW Link. Dataset, Activity Recogniton. MED13, MED12, MED11.

Multiview Extended Video with Activities,
MEVA Test 3:
WWW Link. Information also:
WWW Link. Dataset, Activity Recogniton. Dataset, MEVA. 333 hours of ground-camera and UAV videos and 28 hours of MEVA training Annotations dataset.
See also MEVA: A Large-Scale Multiview, Multimodal Video Dataset for Activity Detection.

PETS 2006 Benchmark Data,
Online2006 BibRef 0600 PETS06
HTML Version. Dataset, Activity Recogniton. Data used at International Workshop on Performance Evaluation of Tracking and Surveillance 2006. BibRef

PETS 2001 Benchmark Data,
Online2001 BibRef 0100 PETS01
WWW Link. Dataset, Activity Recogniton. Data used at International Workshop on Performance Evaluation of Tracking and Surveillance 2001. BibRef

OTCBVS Benchmark Dataset Collection,
OTCBVS072007
WWW Link. Dataset, Activity Recogniton. Beyound the Visual Spectrum (IR especially). Data for various OTCBVS workshops. BibRef 0700

YouTube-8M Dataset,
Labed video dataset.
WWW Link.
WWW Link. Dataset, Video Database. 4700+ visual entities. Introduced in:
See also YouTube-8M: A Large-Scale Video Classification Benchmark.

Abu-El-Haija, S.[Sami], Kothari, N.[Nisarg], Lee, J.[Joonseok], Natsev, P.[Paul], Toderici, G.[George], Varadarajan, B.[Balakrishnan], Vijayanarasimhan, S.[Sudheendra],
YouTube-8M: A Large-Scale Video Classification Benchmark,
Online2016.
DOI Link Paper to introduce
See also YouTube-8M Dataset. BibRef 1600

Boyd, J.E.[Jeffrey E.], Meloche, J.[Jean],
Evaluation of statistical and multiple-hypothesis tracking for video traffic surveillance,
MVA(13), No. 5-6, 2003, pp. 344-351.
WWW Link. 0304
BibRef

Boyd, J.E., Meloche, J., Vardi, Y.,
Statistical Tracking in Video Traffic Surveillance,
ICCV99(163-168).
IEEE DOI BibRef 9900

Oberti, F.[Franco], Stringa, E.[Elena], Vernazza, G.[Gianni],
Performance Evaluation Criterion for Characterizing Video-Surveillance Systems,
RealTimeImg(7), No. 5, October 2001, pp. 457-471.
DOI Link 0110
BibRef

Oberti, F.[Franco], Teschioni, A.[Andrea], Regazzoni, C.S.[Carlo S.],
ROC Curves for Performance Evaluation of Video Sequences Processing Systems for Surveillance Applications,
ICIP99(II:949-953).
IEEE DOI BibRef 9900

Fisher, R.B.[Robert B.],
CAVIAR Test Case Scenarios,
Online BookOctober 2004.
WWW Link. Dataset, Video. From the EC funded CAVIAR project (Context Aware Vision using Image-based Active Recognition). The sequences are labelled (in XML) with both the tracked persons and a semantic description of their activities. 81 video sequences comprising about 90K frames. These sequences include indoor plaza and shopping center observations of individuals and small groups of people walking, browsing, window shopping, fighting, meeting, leaving packages behind, collapsing, entering and exiting shops, etc. BibRef 0410

Optic Flow Data,
Edinburgh2007. Smoothed flow sequences for the Waverly train station scene.
WWW Link. Dataset, Video. Behavior, pedestrian analysis. BibRef 0700

BEHAVE Interactions Test Case Scenarios,
Edinburgh2007. Two views of various scenarios of people acting out various interactions.
WWW Link. Dataset, Video. Behavior, pedestrian analysis. Includes ground truth bounding boxes for much of the data. BibRef 0700

Sigal, L.[Leonid], Balan, A.O.[Alexandru O.], Black, M.J.[Michael J.],
HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion,
IJCV(87), No. 1-2, March 2010, pp. xx-yy.
Springer DOI 1001
Dataset, Human Motion. BibRef
Earlier: A1, A3, Only:
HumanEva: Synchronized Video and Motion Capture Dataset for Evaluation of Articulated Human Motion,
BrownTechnical Report CS-06-08, September 2006.
HTML Version. For the dataset:
HTML Version. Calibrated video sequences synchronized with motion capture data. BibRef

Sanderson, C.[Conrad], Bigdeli, A.[Abbas], Shan, T.[Ting], Chen, S.K.[Shao-Kang], Berglund, E.[Erik], Lovell, B.C.[Brian C.],
Intelligent CCTV for Mass Transport Security: Challenges and Opportunities for Video and Face Processing,
ELCVIA(6), No. 3, December 2007, pp. 30-41.
DOI Link 0801

See also Experimental Analysis of Face Recognition on Still and CCTV Images. BibRef

Norouznezhad, E., Bigdeli, A., Postula, A., Lovell, B.C.,
A high resolution smart camera with GigE Vision extension for surveillance applications,
ICDSC08(1-8).
IEEE DOI 0809
BibRef

Lazarevic-McManus, N., Renno, J.R., Makris, D., Jones, G.A.,
An object-based comparative methodology for motion detection based on the F-Measure,
CVIU(111), No. 1, July 2008, pp. 74-85.
Elsevier DOI 0711
BibRef
Earlier: A1, A2, A4, Only:
Performance evaluation in visual surveillance using the F-measure,
VSSN06(45-52).
WWW Link. 0701
Visual surveillance; Motion detection; Performance evaluation; ROC analysis; F-Measure BibRef

Altun, K.[Kerem], Barshan, B.[Billur], Tuncel, O.[Orkun],
Comparative study on classifying human activities with miniature inertial and magnetic sensors,
PR(43), No. 10, October 2010, pp. 3605-3620.
Elsevier DOI 1007
Inertial sensors; Gyroscope; Accelerometer; Magnetometer; Activity recognition and classification; Feature extraction; Feature reduction; Bayesian decision making; Rule-based algorithm; Decision tree; Least-squares method; k-Nearest neighbor; Dynamic time warping; Support vector machines; Artificial neural networks BibRef

Venetianer, P.L.[Peter L.], Deng, H.L.[Hong-Li],
Performance evaluation of an intelligent video surveillance system: A case study,
CVIU(114), No. 11, November 2010, pp. 1292-1302.
Elsevier DOI 1011
Performance evaluation; Intelligent video surveillance; Embedded vision BibRef

Sasse, M.A.[M. Angela],
Not Seeing the Crime for the Cameras?,
CACM(53), No. 2, February 2010, pp. 22-25.
DOI Link 1101
Why it is difficult - but essential - to monitor the effectiveness of security technologies. BibRef

Fernández Llorca, D.[David], Parra, I.[Ignacio], Sotelo, M.Á.[Miguel Ángel], and Lacey, G.[Gerard],
A vision-based system for automatic hand washing quality assessment,
MVA(22), No. 2, March 2011, pp. 219-234.
WWW Link. 1103
BibRef

Fernández Llorca, D.[David], Vilarino, F., Zhou, J., Lacey, G.,
A multi-class SVM classifier for automatic hand washing quality assessment,
BMVC07(xx-yy).
PDF File. 0709
BibRef

Zhou, J.[Jiang], Vilarino, F.[Fernando], Lacey, G.[Gerard], Li, X.C.[Xu-Chun],
Statistical analysis of ground truth in human labeled data,
IMVIP07(211-211).
IEEE DOI 0709
Analysis of human analysis of hand washing videos. BibRef

Bertuccelli, L.F., Cummings, M.L.,
Operator Choice Modeling for Collaborative UAV Visual Search Tasks,
SMC-A(42), No. 5, September 2012, pp. 1088-1099.
IEEE DOI 1208
Assisting operators in extended searching. Human factors. BibRef

Rathje, J.M., Spence, L.B., Cummings, M.L.,
Human-Automation Collaboration in Occluded Trajectory Smoothing,
HMS(43), No. 2, March 2013, pp. 137-148.
IEEE DOI 1303
BibRef

Muhammad, K.[Khan], Obaidat, M.S.[Mohammad S.], Hussain, T.[Tanveer], del Ser, J.[Javier], Kumar, N.[Neeraj], Tanveer, M.[Mohammad], Doctor, F.[Faiyaz],
Fuzzy Logic in Surveillance Big Video Data Analysis: Comprehensive Review, Challenges, and Research Directions,
Surveys(54), No. 3, May 2021, pp. xx-yy.
DOI Link 2106
video surveillance survey, neural networks, big data, fuzzy tutorial, video summarization, Video surveillance, fuzzy logic BibRef

Liu, S.[Shuai], Huang, S.C.[Shi-Chen], Xu, X.[Xiyu], Lloret, J.[Jaime], Muhammad, K.[Khan],
Efficient Visual Tracking Based on Fuzzy Inference for Intelligent Transportation Systems,
ITS(24), No. 12, December 2023, pp. 15795-15806.
IEEE DOI 2312
BibRef


Deng, A.D.[An-Dong], Yang, T.[Taojiannan], Chen, C.[Chen],
A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition,
ICCV23(20462-20474)
IEEE DOI Code:
WWW Link. 2401
BibRef

Johansen, A.S.[Anders Skaarup], Junior, J.C.S.J.[Julio C. S. Jacques], Nasrollahi, K.[Kamal], Escalera, S.[Sergio], Moeslund, T.B.[Thomas B.],
Chalearn LAP Seasons in Drift Challenge: Dataset, Design and Results,
RealWorld22(755-769).
Springer DOI 2304
Hermal video security monitoring. BibRef

Bolten, T.[Tobias], Pohle-Fröhlich, R.[Regina], Tönnies, K.D.[Klaus D.],
DVS-OUTLAB: A Neuromorphic Event-Based Long Time Monitoring Dataset for Real-World Outdoor Scenarios,
EventVision21(1348-1357)
IEEE DOI 2109
Dataset, Surveilance. Privacy, Rain, Power demand, Neuromorphics, Noise reduction, Pipelines, Vision sensors BibRef

Gowda, S.N.[Shreyank N.], Sevilla-Lara, L.[Laura], Keller, F.[Frank], Rohrbach, M.[Marcus],
CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition,
ECCV22(XX:187-203).
Springer DOI 2211
BibRef

Sevilla-Lara, L.[Laura], Zha, S.X.[Sheng-Xin], Yan, Z.C.[Zhi-Cheng], Goswami, V.[Vedanuj], Feiszli, M.[Matt], Torresani, L.[Lorenzo],
Only Time Can Tell: Discovering Temporal Data for Temporal Modeling,
WACV21(535-544)
IEEE DOI 2106
Training, Visualization, Head, Computational modeling, Motion estimation, Benchmark testing BibRef

Li, L.Z.[Long-Zhen], Nawaz, T.[Tahir], Ferryman, J.M.,
PETS 2015: Datasets and challenge,
AVSS15(1-6)
IEEE DOI 1511
Dataset, PETS 2015. object detection BibRef

Cozzolino, A.[Angelo], Flammini, F.[Francesco], Galli, V.[Valentina], Lamberti, M.[Mariangela], Poggi, G.[Giovanni], Pragliola, C.[Concetta],
Evaluating the Effects of MJPEG Compression on Motion Tracking in Metro Railway Surveillance,
ACIVS12(142-154).
Springer DOI 1209
BibRef

Liu, J.G.[Jin-Gen], Yu, Q.[Qian], Javed, O., Ali, S., Tamrakar, A., Divakaran, A., Cheng, H.[Hui], Sawhney, H.S.,
Video event recognition using concept attributes,
WACV13(339-346).
IEEE DOI 1303
BibRef

Tamrakar, A.[Amir], Ali, S.[Saad], Yu, Q.[Qian], Liu, J.G.[Jin-Gen], Javed, O.[Omar], Divakaran, A.[Ajay], Cheng, H.[Hui], Sawhney, H.S.[Harpreet S.],
Evaluation of low-level features and their combinations for complex event detection in open source videos,
CVPR12(3681-3688).
IEEE DOI 1208
BibRef

Oh, S.M.[Sang-Min], Hoogs, A.J.[Anthony J.], Perera, A.[Amitha], Cuntoor, N.[Naresh], Chen, C.C.[Chia-Chih], Lee, J.T.[Jong Taek], Mukherjee, S.[Saurajit], Aggarwal, J.K., Lee, H.T.[Hyung-Tae], Davis, L.S.[Larry S.], Swears, E.[Eran], Wang, X.Y.[Xiao-Yang], Ji, Q.A.[Qi-Ang], Reddy, K.K.[Kishore K.], Shah, M.[Mubarak], Vondrick, C.[Carl], Pirsiavash, H.[Hamed], Ramanan, D.[Deva], Yuen, J.[Jenny], Torralba, A.B.[Antonio B.], Song, B.[Bi], Fong, A.[Anesco], Roy-Chowdhury, A.K.[Amit K.], Desai, M.[Mita],
A large-scale benchmark dataset for event recognition in surveillance video,
CVPR11(3153-3160).
IEEE DOI 1106
BibRef
And: AVSBS11(527-528).
IEEE DOI 1111
Dataset, Action Recognition. Dataset, Event Recognition. BibRef

Israel, S.A.[Steven A.],
Evaluation of ISR technologies for counter insurgency warfare,
AIPR10(1-5).
IEEE DOI 1010
BibRef

Ryoo, M.S., Chen, C.C.[Chia-Chih], Aggarwal, J.K., Roy-Chowdhury, A.K.[Amit K.],
An Overview of Contest on Semantic Description of Human Activities (SDHA) 2010,
ICPR-Contests10(270-285).
Springer DOI 1008
BibRef

Singh, S., Velastin, S.A., Ragheb, H.,
MuHAVi: A Multicamera Human Action Video Dataset for the Evaluation of Action Recognition Methods,
AVSS10(48-55).
IEEE DOI 1009
BibRef

Desurmont, X., Carincotte, C., Bremond, F.,
Intelligent Video Systems: A Review of Performance Evaluation Metrics that Use Mapping Procedures,
AVSS10(127-134).
IEEE DOI 1009
BibRef

Kuhn, W.[Werner],
A Functional Ontology of Observation and Measurement,
GS09(26-43).
Springer DOI 0912
BibRef

Rose, T.[Travis], Fiscus, J.[Jonathan], Over, P.[Paul], Garofolo, J.[John], Michel, M.[Martial],
The TRECVid 2008 Event Detection evaluation,
WACV09(1-8).
IEEE DOI 0912
BibRef

Ferryman, J.M., Ellis, A.,
PETS2010: Dataset and Challenge,
AVSS10(143-150).
IEEE DOI 1009
BibRef

Ellis, A., Ferryman, J.M.,
PETS2010 and PETS2009 Evaluation of Results Using Individual Ground Truthed Single Views,
AVSS10(135-142).
IEEE DOI 1009
BibRef

Ellis, A., Shahrokni, A., Ferryman, J.M.,
PETS2009 and Winter-PETS 2009 results: A combined evaluation,
PETS-Winter09(1-8).
IEEE DOI 0912
BibRef

Ferryman, J.M., Shahrokni, A.,
PETS2009: Dataset and challenge,
PETS-Winter09(1-6).
IEEE DOI 0912
BibRef

Kovesi, P.[Peter],
Video Surveillance: Legally Blind?,
DICTA09(204-211).
IEEE DOI 0912
Surveillance cameras are not very good. BibRef

Wang, H.[Heng], Ullah, M.M.[Muhammad Muneeb], Klaser, A.[Alexander], Laptev, I.[Ivan], Schmid, C.[Cordelia],
Evaluation of local spatio-temporal features for action recognition,
BMVC09(xx-yy).
PDF File. 0909
STIP features. Code, STIP.
WWW Link. BibRef

Sulman, N.[Noah], Sanocki, T.A.[Thomas A.], Goldgof, D.[Dmitry], Kasturi, R.[Rangachar],
How effective is human video surveillance performance?,
ICPR08(1-3).
IEEE DOI 0812
BibRef

Vezzani, R.[Roberto], Cucchiara, R.[Rita],
ViSOR: Video Surveillance Online Repository,
BMVA(2010), No. 2, 2010, pp. 1-13.
PDF File. 1209
BibRef
Earlier:
Annotation Collection and Online Performance Evaluation for Video Surveillance: The ViSOR Project,
AVSBS08(227-234).
IEEE DOI 0809
BibRef

Roth, D., Koller-Meier, E., Rowe, D., Moeslund, T.B., Van Gool, L.J.,
Event-Based Tracking Evaluation Metric,
Motion08(1-8).
IEEE DOI 0801
BibRef

Nghiem, A.T., Bremond, F., Thonnat, M., Valentin, V.,
ETISEO, performance evaluation for video surveillance systems,
AVSBS07(476-481).
IEEE DOI 0709
BibRef

Garofolo, J.[John],
Directions in automatic video analysis evaluations at NIST,
AVSBS07(6-6).
IEEE DOI 0709
Evaluation, Video Analysis. BibRef

Taylor, G.R.[Geoffrey R.], Chosak, A.J.[Andrew J.], Brewer, P.C.[Paul C.],
OVVV: Using Virtual Worlds to Design and Evaluate Surveillance Systems,
VS07(1-8).
IEEE DOI 0706
BibRef

Negre, A., Tran, H., Gourier, N., Hall, D., Lux, A., Crowley, J.L.,
Comparative Study of People Detection in Surveillance Scenes,
SSPR06(100-108).
Springer DOI 0608
BibRef

Nghiem, A.T., Bremond, F., Thonnat, M., Ma, R.,
A New Evaluation Approach for Video Processing Algorithms,
Motion07(15-15).
IEEE DOI 0702
To avoid dataset dependence. The maximum difficulty level of the videos at which the algorithm is performing good enough is defined as the upper bound of the algorithm capacity for handling the problem. BibRef

Tsuchiya, M.[Masamitsu], Fujiyoshi, H.[Hironobu],
Evaluating Feature Importance for Object Classification in Visual Surveillance,
ICPR06(II: 978-981).
IEEE DOI 0609
BibRef

Lienhart, R.[Rainer],
Algorithm competition,
VSSN05(53-54).
WWW Link. 0511
Evaluation discussion. For surveillance applications. BibRef

List, T., Bins, J., Vazquez, J., Fisher, R.B.,
Performance evaluating the evaluator,
PETS05(129-136).
IEEE DOI 0602
How to compare to human results. BibRef

Annesley, J., Orwell, J., Renno, J.R.,
Evaluation of MPEG7 color descriptors for visual surveillance retrieval,
PETS05(105-112).
IEEE DOI 0602
BibRef

Muller-Schneiders, S., Jager, T., Loos, H.S., Niem, W.,
Performance evaluation of a real time video surveillance system,
PETS05(137-143).
IEEE DOI 0602
BibRef

Ziliani, F., Velastin, S.A., Porikli, F.M., Marcenaro, L., Kelliher, T., Cavallaro, A., Bruneaut, P.,
Performance Evaluation of Event Detection Solutions: The CREDS Experience,
AVSBS05(201-206).
IEEE DOI 0602
BibRef

Zang, Q.[Qi], Klette, R.[Reinhard],
Object Classification and Tracking in Video Surveillance,
CAIP03(198-205).
Springer DOI 0311
BibRef

Zang, Q.[Qi], Klette, R.[Reinhard],
Evaluation of an Adaptive Composite Gaussian Model in Video Surveillance,
CAIP03(165-172).
Springer DOI 0311
BibRef

Jaynes, C., Webb, S., Steele, R.M., Xiong, Q.,
An Open Development Environment for Evaluation of Video Surveillance Systems,
PETS02(32-39). 0207
BibRef

Ferryman, J.M.,
Performance Evaluation of Tracking and Surveillance,
EEMCV01(xx-yy). 0110
BibRef

Ellis, T.,
Performance Metrics and Methods for Tracking in Surveillance,
PETS02(26-31). 0207
BibRef

Chapter on Motion -- Feature-Based, Long Range, Motion and Structure Estimates, Tracking, Surveillance, Activities continues in
Vehicle Motion Understanding and Analysis .


Last update:Apr 18, 2024 at 11:38:49