CS672 - Computer System Performance Evaluation - Spring 2008

George Mason University
DEPARTMENT OF COMPUTER SCIENCE
Mondayss 4:30pm to 7:10pm Science and Tech I, Room 126
Also taught via the Internet at the same time. Enroll in section 002. Contact the CS department for details.
Professor Daniel A. Menasce'
Course office hours Mondays 3:00pm to 4:00 PM or by appt through Laura Harrison (lharrisc at gmu dot edu); e-mail at all times
Phone: 703-993-1537
E-mail: menasce at cs dot gmu dot edu (please prefix the subject of your message with CS672)

CS 672 Home Page
 

DESCRIPTION :

CS672 is a 3-credit course with CS571 as prerequisite. It introduces the main concepts and techniques needed to plan the capacity of computer systems, predict their future performance under different configurations, and design new applications that meet performance requirements. The course is mainly based on the use of analytic queuing network models of computer systems. These techniques are applied to study the performance of centralized, distributed, parallel, client/server systems, Web server and e-commerce site performance. The course also discusses performance measuring tools for operating systems such as Unix and Windows NT. The course provides the students with hands-on experience in performance evaluation through a project. The concept and applications of software performance engineering are also covered.

Back to the top


 
 

GRADING POLICY:

Grades are based on a midterm, a take-home final, and one project. Grades will be numerical on the scale 0-100. Your final numerical grade, G, is computed as:
G = 0.35* Midterm + 0.30 * Final + 0.35 * Project.

The following table is used to convert your numerical grade G to a letter grade:

G  letter grade
[97,100]  A+
[93,97)  A
[88,93)  A-
[85,88)  B+
[81,85)  B
[77,81)  B-
[65, 77)  C
< 65  F

There is no curving. No extra credit assignments will be given after the semester is over to increase grades. The instructor may decide to give an extra-credit assignment during the semester. Everyone will be given the opportunity to do the extra-credit assignment. Several homeworks will be assigned during the semester but will not considered for the final grade. However, students who turn in all homeworks in time and get most of them correct may receive an upgrade in their letter grade if their numerical grade G is very close to the boundary between two letter grades.

Back to the top

SYLLABUS:
Week 1:  Computer System Lifecycle and From Systems to Descriptive Models
Week 2: Quantifying Performance Models
Weeks 3-4: Performance Engineering Methodology and Case Study I: A Database Service
Week 5: Multiclass Open Queuing Network Models
Weeks 6-7: Case Study II: A Web Server and Multiclass Closed Queuing Network Models
Week 8: Midterm
Week 9: Case Study III: A Data Center
Week 10: Markov Models
Week 11: Case Study IV: An E-Business Service and Case Study V: A Help-Desk Service
Week 12: Queuing Networks with Load Dependent Devices.
Week 13: Non-Product Form Queuing Network Models
Back "to the top

READINGS:

Required text books:

  1. D. A. Menascé, V. Almeida, and Larry W. Dowdy, Performance by Design: Computer Capacity Planning by Example, Prentice Hall, 2004.
  2. D. A. Menascé and V. Almeida, Capacity Planning for Web Services: Metrics, Models, and Methods, Prentice Hall, 2002.
  3. Slides by D. A. Menascé.
Other recommended books:
  1. Introduction to Computer System Performance Evaluation, K. Kant, McGraw Hill, 1992.
  2. Probability, Stochastic Processes, and Queuing Theory, Randolph Nelson, Springer Verlag, 1995.
  3. Quantitative System Performance, E. Lazowska, J. Zahorjan, G. Graham, and K. Sevcik, Prentice Hall, 1984.
  4. The Benchmark Handbook for Database and Transaction Processing Systems, ed. Jim Gray, Morgan Kauffman, 1991.
  5. Optimizing Oracle Performance: A Practitioner's Guide to Optimizing Response Time, Cary Milsap, 2003, O'Reilly.
  6. Configuration and Capacity Planning for Solaris Servers, Brian Wong, Prentice Hall, 1997.
  7. Windows 2000 Performance Guide, M. Friedman and O. Pentakalos, O'Reilly, 2002.
  8. Sun Performance and Tuning, second edition, Adrian Cockcroft and Richard Pettit,Prentice Hall, 1998.
  9. Simulation Modeling and Analysis, A. M. Law and W. D. Kelton, 3rd edition, McGraw Hill,200.
  10. Simulation with Arena, W. David Kelton, Randall P. Sadowski, Deborah A. Sadowski, McGraw Hill, 1998.
Other sources of relevant material:
  1. Proceedings of the ACM Sigmetrics Conference.
  2. Proceedings of the Computer Measurement Group (CMG) Conference Proceedings.
Back to the top

RESOURCES

  • Transaction Processing Performance Council: organization that defines and publishes transaction processing and database benchmarks known as the TPC benchmarks.
  • The Standard Performance Evaluation Corporation: organization that establishes, maintains, and endorses a standardized set of benchmark metrics for performance evaluation known as SPEC ratings. They are also responsible for the SPECWeb benchmark for Web servers.
  • WebStone Benchmark: a benchmark, orginally developed by SGI, used to assess Web servers.
  • Keynote:a service that measures end-user performance for E-commerce sites.
  • The WEBBIB Project: an online bibliography of distributed systems topics related to the WWW.
  • ACM Digital Library: look for the Proceedings of the ACM Sigmetrics conferences, as well as papers in other ACM conferences and journals.
  • IEEE/IEE Eletronic Library: look for papers in the IEEE Tr. Computers, IEEE Tr. Software Engineering, IEEE Transactions on Parallel and Distributed Systems, Conference Proceedings, as well as in various IEEE magazines such as IEEE Internet Computing.
  • Performance Evaluation Journal: Go to the Elsevier link on this page and select the Performance Evaluation journal.


    • Back to the top
     
     

    PROJECT:

    Students will work individually or in groups of two. Each team or student will work on a different project and will make a presentation (a final exhibition) to the whole class. The project presentation is scheduled for the day of the final. Please mark your calendars now. Each group has to turn in a written report describing the performance problem being studied, the model built, the data collection procedures used, the various numerical results, and conclusions obtained. Each member of the group must turn in an individual (one-page) report indicating his/her perception of how the work was distributed among the group members and what the contribution of each was. Students are encouraged to report any breakdowns on group organization and operational aspects as early as possible, so that these problems can be overcome. In order for projects to complete successfully, it is very important to plan all activities and to follow a schedule. Starting to work on a project a few days before it is due will very likely cause problems to its successful completion. A Web page will be developed for the group projects and students are expected to "publish" their projects in html.

    Back to the top

    IMPORTANT DATES :
    First Day of Classes  January 28, 2008. 
    Project Proposal Due March 3, 2008
    Spring break (no classes)  March 10, 2008. 
    Midterm  March 31, 2008. 
    Last day of classes May 5, 2008.
    Final exam available  April 21, 2008.
    Final due date:  May 5, 2008.
    Project presentations  May 5, 2008.
    Back to the top

    IMPORTANT RULES :

    No collaboration is allowed among students in any of the individual exams. Students are allowed to discuss with other students the solution of homework assignments. Collaboration in the project is only allowed within the group.

    Back to the top

    Last updated: January 7, 2008.