锐客网

  1. 首页 > it技术 > >

Web Server 工作流

【Web Server 工作流】Web Server Workload Characterization
Assignment 3 and 4, COMPSCI 315
Due: Refer to the deadline on Canvas; Submission via Canvas
1 Introduction >Internet traffic measurement involves collecting network data that can be analyzed for several purposes such
as traffic modeling, designing better network protocols, and traffic management. The growth in popularity
of Web in the 1990s resulted in researchers trying to characterize Web traffic. These research works have
utilized Web server logs to understand the workload characteristics of Web servers. The results of the
research has led to improving performance of Web applications, designing better caching and load balancing
techniques, and providing better user experience to clients, among other things [1, 3–5].
2 Web Server Access Logs
In this assignment, you will analyze one of two university Web server access logs [2]:

  1. A campus-wide Web server at the University of Saskatchewan (UofS_access_log).
  2. A department-level Web server at the University of Calgary (UofC_access_log).
    The server access log contains information about all requests made to the server and the corresponding
    server responses. The server log is in the fixed text-based Common Log Format and has the following syntax:
    hostname - - [dd/mm/yyyy:hh:mm:ss time_zone] object response_code transfer_size
    The hostname is the resolved name or IP address of the client making a request for an object stored on
    the Web server. The following fields (- -) are usually empty, but may contain user-identifier information
    such as username. The next field indicates the day and time the request was made along with the time zone.
    The URL requested is noted in the object field. The response_code field indicates the HTTP response
    code returned by the server. The transfer_size field records the bytes transferred by the server.
    For example, the following is a line from the UofS_access_log:
    imhotep.usask.ca - - [15/Sep/1995:16:02:09 -0600] "GET /changes.html HTTP/1.0" 200 1254
    This line represents a request made by host imhotep.usask.ca on September 15, 1995 at 4:02:09 p.m.
    The time zone is central time (GMT-0600). The host requested the HTML file called changes.html using
    HTTP version 1.0. This request was successfully completed by the server as shown by the status code 200.
    The server transferred 1254 bytes to the host imhotep.usask.ca.
    Note the following about the datasets:
    UofS_access_log: This trace contains seven month’s worth of all HTTP requests to the university
    Web server. The logs fully preserve the originating host and HTTP request. A local client is one
    containing usask.ca in the hostname or an IP address with 128.233.X.X. All others are considered
    remote clients. Timestamps have 1 second resolution.
    1
    UofC_access_log: This trace contains approximately one year’s worth of all HTTP requests to the
    University of Calgary’s Department of Computer Science Web server. The hosts making requests to
    the server have had their addresses removed to preserve privacy. Hosts are identified as either local
    or remote where local is a host from the University of Calgary, and remote is a host from outside of
    the University of Calgary domain. Paths have been removed. Files were numbered from 1 for the
    first file encountered in the trace. Files retain the original file extension, so that the type of file can
    be determined. Paths of the filenames have been removed. Modified filenames consist of two parts:
    num.type, where num is a unique integer identifier, and type is the extension of the requested file.
    Timestamps have 1 second resolution.
  3. Web Server Workload Analysis
    Choose one dataset you like and answer all questions:
  4. Based on your learning of Internet measurements in this course answer the following questions:
    (a) What measurement mechanism was used for the collection of the Web server logs? Active, Passive,
    or both.
    (b) What type of network were the measurements taken from? Edge network, core network, or both.
    (c) What type of analysis techniques did you apply on the dataset to get the answers? Online, offline,
    or both.
    (d) Is analyzing server logs the only way to characterize the workload of a Web server? Why?
  5. How many requests are made per day on average?
  6. How many bytes were transferred during the entire log duration expressed in Megabytes (MB)?
  7. What is the average number of bytes transferred per day expressed in MB per day?
  8. Produce a breakdown of the server response codes expressed in percentage of the total number of
    requests. Group the status code as follows: Successful, Not Modified, Found, Unsuccessful. A successful
    response (status code: 200) means that the server received a request for a valid object (for which the
    client has the necessary access privilege), the object was found, and returned successfully to the client.
    A not modified response (status code: 304) means that the client already has a copy of the requested
    object in its cache, wants to verify if the object is up-to-date, and the client is informed that the
    object has not been modified at the server. A found response (status code: 302) results when the
    requested object is known to be stored in a different location than the URL requested by the client.
    The server responds with the new URL in this situation. A unsuccessful response (status code: 4XX
    and 5XX) happens when the requested object does not exist on the server, the client did not have
    access permission, or there was a server-side error.
  9. How many requests are made by local clients and remote clients, respectively? Report your answer as
    a percentage of total requests.
  10. How many bytes are transferred by local clients and remote clients, respectively? Report your answer
    as a percentage of total bytes transferred.
  11. Produce a breakdown of requests by file type category. The file categories are as follows: Video, Sound,
    Dynamic, Formatted, HTML, Images, Others. Report your answer as a percentage of total requests.
    The file categories by file extensions are described in Table 1.
  12. Using Table 1, produce a breakdown of bytes transferred by each file category. Report your answer as
    a percentage of total bytes transferred.
  13. Using Table 1, calculate the average transfer sizes (in bytes) of each file category.
    2
    Table 1: File categories
    Category File extension
    HTML html, htm, shtml, map
    Images gif, jpeg, jpg, xbm, bmp, rgb, xpm
    Sound au, snd, wav, mid, midi, lha, aif, aiff
    Video mov, movie, avi, qt, mpeg, mpg
    Formatted ps, eps, doc, dvi, txt
    Dynamic cgi, pl, cgi-bin
    Others Everything else
  14. Identify all unique object requests in the log and sort them based on frequency. Next, identify all the
    objects that were requested only once in the log. What percentage of unique objects are accessed only
    once in the log? What percentage of bytes are accessed only once in the log?
  15. Produce a Cumulative Distribution Function (CDF) plot of the transfer sizes of all distinct objects.
    The x-axis should be in log-10 scale.
  16. Produce at least one plot to show the percentage of total requests per hour of the day, the percentage
    of total requests per day of the week, or percentage of total requests per month of the year.
  17. Produce a CDF plot of the inter-reference times of objects that are requested more than once. The
    x-axis should be in log-10 scale.
    For questions 6 onwards, your analysis should be based on successful requests only. Report
    your results to 2 decimal places. Some requests in the log may be malformed. It is safe to
    ignore these requests in your analysis. Please check that these requests account for a negligible
    fraction of the total requests. Briefly comment on your results (Explain the results and discuss
    their implications).
  18. Submission
    Complete the attached file with your answers. Convert this file to PDF format. Rename it
    to your username.pdf. Submit the file to Canvas. Submit answers to questions 1-7 as part of
    Assignment 3. Submit answers to questions 8-14 as part of Assignment 4.
    A code template will be provided at a later date. For added challenge, you are encouraged to write the
    parser and analysis scripts from scratch using a programming language of your choice. You are free to use
    online resources (e.g., online code, tools) as long as you provide appropriate attribution. You do not need
    to submit the code. You should keep the code, in case we wish to see it. You are encouraged to discuss the
    assignment with each other, however, the code and the produced results must be done individually.
    Questions regarding this assignment and code template should be directed to the course tutors.
  19. Grading Scheme
    Each question is worth 10 points. Assignment 3 is worth 70 points. Assignment 4 is worth 70 points. For
    each question, you will receive full points for the correct answer. You will receive 50% points for an answer,
    which is close to the correct answer. You will receive zero points for an answer that is far off from the correct
    answer. Each answer should have a brief explanation to receive full marks.
    References
    [1] Martin Arlitt and Tai Jin, A Workload Characterization Study of the 1998 World Cup Web Site, IEEE
    Network 14 (2000), no. 3.
    3
    [2] Martin Arlitt and Carey Williamson, Internet Web Servers: Workload Characterization and Performance
    Implications, IEEE/ACM Trans. Netw. 5 (1997), no. 5, 631–645.
    [3] Leeann Bent, Michael Rabinovich, Geoffrey M. Voelker, and Zhen Xiao, Characterization of a Large Web
    Site Population with Implications for Content Delivery, WWW 9 (2006), no. 4.
    [4] Venkata Padmanabhan and Lili Qiu, The Content and Access Dynamics of a Busy Web Site: Findings
    and Implications, Proc. ACM SIGCOMM, 2000.
    [5] Weisong Shi, Y Wright, Eli Collins, and Vijay Karamcheti, Workload Characterization of a Personalized
    Web Site and its Implications for Dynamic Content Caching, Proc. WCW, 2002.

    推荐阅读


    上一篇：随笔。

    下一篇：C++中新手容易犯的十种编程错误汇总