Multimedia and Communications

classroom: R108 of technology building III, Tuesday 9:10 ~12:00 a.m.

Associate Professor Herng-Yow Chen
Email: hychen@csie.ncnu.edu.tw,
phone: 049-2910960#4843
office: R401, technology building III

Lectures ||  Resource ||  Homework

---

Course Objective

1. This course investigates multimedia (speech, audio, and video) compression; multimedia processing and retrieval; and relevant important issues on communication.
2. By the end of semester, you should be able to master the technical underpinnings of multimedia system and related communication applications.
3. Students should learn to present paper or their own work.

Evaluation

• Class participation: 10%, Assignments: 30%, Midterm (oral report): 30%,
• Final exam (project presentation): 30%, the topics & schedules (期末簡報主題與順序)
• Important! 期末簡報檔 請於 6/23 （一）前寄給 吳文凱 (Kevin)助教 (email: s96321533@ncnu.edu.tw)

Homework regulations

• In addition to print-out version, you should email me your report and the paper (or article) you choose before deadline. The email subject:  multimedia, homework#, your_student-id
• All reports should be accessible via your web entry.

http://www.ncnu.edu.tw/~your_account/multimedia

References

1. IEEE Multimedia Magazine (paper reading)
2. IEEE Internet Computing Magazine (paper reading)
3. IEEE Communication (paper reading)
4. other relevant papers selected from IEEE, ACM multimedia conference, journal, and magazine
5. Multimedia: Computing, Communications, and Applications, Ralf Steinmetz and Klara Nahrstedt, Prentice Hall, 1995, ISBN 0-13-324435-0
6. Understanding Networked Multimedia applications and technology, Francois Fluckiger, Prentice Hall, 1995, ISBN 0-13-190992-4
7. Handbook of Multimedia Computing, Borko Furht (editor-in-chief), CRC press, 1999, ISBN 0-8493-1825-4
8. Handbook of Internet and Multimedia: Systems and Applications, Borko Furht (editor-in-chief), CRC press, 1999, ISBN 0-8493-1858-0

Course Schedule

1. Introduction to Multimedia and Communication
2. Multimedia System Overview (paper study)
3. Multimedia Communication (paper study)
4. Speech and Audio Compression
5. Sampling and Interpolation
6. Quantization
7. Audio Coding
8. MPEG Audio Coding (paper study)
9. *** Midterm *** (paper oral report)
10. Graphics and Image Compression
11. Digital Image Processing, Basics of Color Images
12. Image Filtering, Image Resizing
13. Transform Coding, Discrete Cosine Transform (DCT)
14. Image Coding Standards: JPEG, JPEG2000
15. Video Compression
16. Motion Estimation and Compensation: MPEG-1, MPEG-2, MPEG-4
17. Multimedia Systems & Information Retrieval
18. Final examination (project report)

 Lecture slides

        Use these as study guides--not as an excuse to skip class. I may be changing/refining these files right up until class time!

• Lecture 1:  Introdcution to Multimedia and Communication. open, talk in csie, intro, intro2(rowe)

Reading: (print this) (reading assignment)

Borko Furht, "Multimedia Systems: An Overviews," IEEE Multimedia Magazine, pp. 47-59, Spring 1994.

Reading:

Lars C. Wolf, Carsten Griwodz, and Ralf Steinmetz, "Multimedia Communication," Proceedings of the IEEE, Vol. 85, No. 12, pp. 1915-1993, December 1997.

Reading:

Jayant, N.; Johnston, J.; Safranek, R., "Signal compression based on models of human perception," Proceedings of the IEEE, Volume: 81,   Issue: 10,   Oct 1993, Page(s): 1385-1422.

• Lecture 2:  Speech and Audio Compression
• Sampling and Interpolation (refer to http://eeweb.poly.edu/~yao/EE3414/sampling.pdf) ,
• Quantization (refer to http://eeweb.poly.edu/~yao/EE3414/quantization.pdf)
• Audio Coding (refer to http://eeweb.poly.edu/~yao/EE3414/audio_coding.pdf)
• Lecture 8: Audio Compression & Coding

Reading:

Fundamental of Psychoacoustics, http://sound.eti.pg.gda.pl/SRS/psychoacoust.html

Reading:

Zwicker E., Zwicker T., "Audio Engineering and Psychoacoustics: Matching Signals to the Final Receiver, the Human Auditory System", Journal of Audio Engineering Society, vol. 39, No. 3, March 1991, pp. 115-126.

Reading:

Painter and Spanias, " Perceptual Coding of Digital Audio", Proc. IEEE, April 2000.

Spanias, A. S. (1994). Speech Coding: A Tutorial Review.

Portions published in Proceedings of the IEEE, Oct. 1994.

Allen Gersho, "Advances in Speech and Audio Compression," Proceedings of the IEEE , Volume: 82 Issue: 6 , Jun 1994, Page(s): 900 -918.

Reading:

Davis Pan, "A tutorial on MPEG/audio compression," Multimedia, IEEE , Volume: 2 Issue: 2 , Summer 1995 , Page(s): 60 -74.

Reading: (print this)

Peter Noll, "MPEG Digital Audio Coding Standards," 2000 CRC Press LLC, http://www.ff.vu.lt/studentams/tekstai/vizualizavimas/mpeg%20audio%20coding.pdf

Reading:

Eric D. Scheirer, "Structured audio and effects processing in the MPEG-4 multimedia standard," Multimedia System, Vol 7, Page(s): 11-22, 1999.

Reading:

Chris Kyriakakis, "Fundamental and technological limitations of immersive audio systems," Proceedings of the IEEE , Volume: 86 Issue: 5 , May 1998, Page(s): 941 -951.

• Lecture 3: Graphics and Image Compression
• Video formation, perception, and representation (from Textbook Video Processing and Communications ) (draft chapter dowload) (Chapter 1 slides: Basics of Video)
• Basics of color imaging (http://eeweb.poly.edu/~yao/EE3414/image_color.pdf)
• Image Filtering (http://eeweb.poly.edu/~yao/EE3414/image_filtering.pdf)
• Image coding standards: JPEG and JPEG2000 (http://eeweb.poly.edu/~yao/EE3414/JPEG.pdf)

Reading: (reading assignment)

Gregory K. Wallace, "The JPEG still picture compression standard," Communications of the ACM, Volume: 34, Issue 4 (April 1991), Page(s): 30 - 44.  You can access the revised version of the same paper (clearer for print) published in IEEE Transaction on Consumer Electronic. 1991.

Reading: (print this)

A. Skodras, C. Christopoulos, T. Ebrahimi, "The JPEG2000 Still Image Compression Standard", IEEE Signal Processing Magazine, pp. 36-58, Sept. 2001.

• Lecture 4 : Video Compression

• Image and Video Compression  Learning Tool, http://www-it.et.tudelft.nl/~inald/vcdemo/
• MPEG Video Coding I- MPEG 1, and 2
• MPEG Video Coding II- MPEG-4, 7, and Beyond

Reading:

 Didier Le Gall, "MPEG: a video compression standard for multimedia applications," Communications of the ACM, Volume: 34, Issue 4 (April 1991), Page(s): 46 - 58.

Reading:

Multimedia compression techniques and standards: JPEG and MPEG

Reading:

B.G. Haskell, P.G. Howard, Y.A. LeCun, A. Puri, J. Ostermann, M.R. Civanlar, L. Rabiner, L. Bottou, and P. Haffner, "Image and Video Coding -- Emerging Standards and Beyond," IEEE Transactions on Circuits and Systems for Video Technology, vol. 8, no. 7, pp. 814-837. 1998.

Reading:

Thomas Sikora, "MPEG Digital Video-Coding Standards," IEEE Signal Processing Magazine, Volume: 14 Issue: 5 , Sept. 1997, Page(s): 82 -100.

• Lecture 5: Audio Retrieval and Navigation Interfaces

Reading:

Asif Ghias , Jonathan Logan , David Chamberlin , Brian C. Smith, "Query by humming: musical information retrieval in an audio database," Proceedings of the third ACM international conference on Multimedia, January 1995, page(s) 231-236.

Reading:

Wold, E.; Blum, T.; Keislar, D.; Wheaten, J., "Content-based classification, search, and retrieval of audio," Multimedia, IEEE , Volume: 3 Issue: 3 , Fall 1996, Page(s): 27 -36.

Reading:

Lonce Wyse, Stephen W. Smoliar, "Toward content-based audio indexing and retrieval and a new speaker discrimination technique," Computational Auditory Scene Analysis: Proceedings of the IJCAI-95 Workshop. Erlbaum (1998), Page(s): 351-360.

Reading:

M. G. Brown, J. T. Foote, G. J. F. Jones, K. Sparck Jones, S. J. Young, "Open-vocabulary speech indexing for voice and video mail retrieval," Proceedings of the fourth ACM international conference on Multimedia, 1996, page(s) 307-316. (best paper award)

Reading:

Barry Arons, "SpeechSkimmer: a system for interactively skimming recorded speech,"  ACM Transactions on Computer-Human Interaction (TOCHI) March 1997.
.

• Lecture 6: Video Analysis, Structuring, Indexing, and Browsing

Reading:

Tonomura, Y.; Akutsu, A.; Taniguchi, Y.; Suzuki, G., "Structured Video Computing," Multimedia, IEEE , Volume: 1 Issue: 3 , Autumn/Fall 1994, Page(s): 34-

Reading:

Minerva Yeung; Boon-Lock Yeo; Bede Liu, "Extracting story units from long programs for video browsing and navigation," Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems, 1996, Page(s): 296 -305.

Reading:

Smith, M.A.; Kanade, T.,"Video skimming and characterization through the combination of image and language understanding techniques

," Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, 1997, Page(s): 775 -781.

• Lecture 7: Special Issues on MPEG-7, paper readings selected from IEEE Circuits and Systems for Video Technology, Volume: 11,   Issue: 6,   Jun 2001
• Lecture  : Hardware support for Multimedia, Storage
• Lecture :  Server design for Multimedia

Reading:  Chen at al, "RAID: High-Performance, Reliable Secondary Storage," ACM Computing Surveys, Vol. 26, No.2, June 1994, Page(s): Pages: 145 - 185. (homework due: May 13th)

• Lecture: Quality of Service, Operating System Support
• Lecture: Multimedia Synchronization

Reading:   Steinmetz R (1996) Human Perception of Jitter and Media Synchronization. IEEE journal on Selected Areas in Communications, 14(1), pp. 61-72.
Reading:   Blakowski, G.; Steinmetz, R, (1996) A media synchronization survey: reference model, specification, and case studies. IEEE journal on Selected Areas in Communications, 14(1), pp. 5-35.

Topic Presentation (by 2004 students)

• Presentation 12 (May 27th) Presenter Man-Hua Wu, Topic: Spoken Dialogue Technology
• "Spoken Dialogue Technology:Enabling the Conversational User Interface", ACM Computing Surveys (CSUR) March 2002, Volume 34, Issue 1,  pp. 90-169.
• Presentation 11 (May 20th) Presenter Hao-Tung Lin, Topic: News on Demand, Spoken Document Indexing
• "Transcribing broadcast news for audio and video indexing," Communications of the ACM, Volume 43 ,  Issue 2  (February 2000), pp. 64-70.
• "Machine learning of event segmentation for news on demand," Communications of the ACM, Volume 43 ,  Issue 2  (February 2000), pp. 35-41.
• "News on demand introduction," Communications of the ACM, Volume 43 ,  Issue 2  (February 2000), pp. 32-34.
• "Integrated technologies for indexing spoken language," Communications of the ACM, Volume 43 ,  Issue 2  (February 2000), pp. 48-56.
• Presentation 10 (May 13th) " Presenter Kuo-Yu Liu, Topic: eLearning system survey: Integration of Web and Multimedia Technologies
• "Web-Based Multimedia Tools for Sharing Educational Resources," ACM Journal of Educational Resources in Computing, Vol. 1, No. 1, Spring 2001.
• "A Web-based Synchronized Multimedia System for Distance Education," Proceedings of the 2001 ACM symposium on Applied computing, pp. 94 - 98.
• "An RTP-based synchronized hypermedia live lecture system for distance education," Proceedings of the seventh ACM international conference on Multimedia, October 1999, pp. 91 - 99.
• "Classroom 2000: An experiment with the instrumentation of a living educational environment," IBM System Journal, Vol 38, No 4, pp. 508-530, 1999.
• "The Web and distance learning what is appropriate and what is not," ACM SIGCUE Outlook October 1997, Volume 25 Issue 4.

"Teaching and learning as multimedia authoring the classroom 2000 project," Proceedings of the fourth ACM international conference on Multimedia,
1996, pp. 187 - 198.

• Presentation 9 (May 6th): Presenter Han-Li Cheng,
• "Internet Telephony, Architecture and Protocols: an IETF Perspective,"  Computer Networks and ISDN Systems, Febuary 11, 1999, vol 31/3, pp. 237-255.
• " Internet Telephony Gateway Location," Proceedings of IEEE Infocom, 1998.
• Presentation 8 (April 29th): Presenter David Chang,
• "Synchronization in Multimedia Data Retrieval," International Journal of Network Management January, Volume 7, Issue 1, pp. 33-62, 1997.
• Presentation 7 (April 22th): Presenter Yung-Chuan Liao,
• "The JPEG2000 Still Image Coding System:  An overview," IEEE Transactions on Consumer Electronics, Vol. 46, No. 4, pp. 1103-1127, November 2000.
• "Image coding with the Wavelet transform," IEEE Transaction on Image Processing, vol. 1, no. 2, pp. 205--220, Feb. 1992.
• Presentation 6 (April 15th): Presenter Ing-Jay Chang,
• "Storage and Retrieval of Feature Data for a Very Large Online Image Collectionf", IEEE Bulletin of the Technical Commitee on Data Engineering, Page(s) 19-27, Vol. 19, No. 4, December 1996.
• "Color and Texture Descriptors,"  IEEE Circuits and Systems for Video Technology, Page(s): 703-715, Volume: 11,   Issue: 6,   Jun 2001.
• "MPEG-7 Visual Shape Descriptors,"  IEEE Circuits and Systems for Video Technology, Page(s): 716-719, Volume: 11,   Issue: 6,   Jun 2001.
• Presentation 5 (April 8th): Presenter C.-W. Tien,
• "SMIL 2.0: XML for Web multimedia, Internet Computing, IEEE, Volume: 5 Issue: 5 , Sep/Oct 2001, Page(s): 78 -84.
• "SMIL 2.0 part 1: Overview, concepts, and structure," IEEE Multimedia Magazine, Page(s): 82-88, Volume: 8, Issue: 4, Oct-Dec 2001.
• "SMIL 2.0 part 2: Examples and comparisons," IEEE Multimedia Magazine, Page(s): 74-84, Volume: 9,   Issue: 1,   Jan-Mar 2002.
• Presentation 4 (March 24th): Presenter Sheng-Hao Hung,
• "Everything you wanted to know about MPEG-7: part 1," IEEE Multimedia Magazine, Page(s): 65-77, Volume: 6,   Issue: 3,   Jul-Sep 1999.
• "Everything you wanted to know about MPEG-7: part 2," IEEE Multimedia Magazine, Page(s): 64-73, Volume: 6,   Issue: 4,   Oct-Dec 1999.
• Presentation 3 (March 17th): Presenter Hao-Tung Lin,
• "Applications of video-content analysis and retrieval," IEEE Multimedia Magazine, Page(s): 42-55, Volume: 9,   Issue: 3,   Jul-Sep 2002..
• Presentation 2 (March 11th): Presenter Man-Hua Wu,
• "Adaptive Video Multicast over the Internet," IEEE Multimedia Magazine, Page(s): 22-33, Volume: 10,   Issue: 1,   Jan-Mar 2003.
• Presentation 1 (March 4th): Presenter Kuo-Yu Liu,
• "Multimedia Standards building blocks of the Web," IEEE Multimedia Magazine, Page(s): 13-15, Volume: 8,   Issue: 3,   Jul-Sep 2001.

Software

• For paper reading (.pdf), please download Adobe Acrobat Reader

Resource

• Facsimile Theory
• JPEG
• MPEG
• RAID edu.

References:

1. TCP Congestion control
2. SNR and PSNR
3. Image and Video Compression  Learning Tool, http://www-it.et.tudelft.nl/~inald/vcdemo/
4. Special Issues on MPEG-7, IEEE Circuits and Systems for Video Technology, Volume: 11,   Issue: 6,   Jun 2001
5. http://www.williamson-labs.com