A Comprehensive Guide to HTTP Archive (HAR) Analysis for Debugging Web Application Performance

Analyzing HTTP Archive files has become an invaluable technique for diagnosing those hard-to-pinpoint web application issues that impact page load times, render performance, and overall user experience.

With over 53% of users expecting pages to load in under 3 seconds, and 44% likely to abandon a page taking over 10 seconds (Google), web developers constantly grapple with performance tuning. But addressing performance complaints by users requires drilling down into granular network activity data that HAR files aim to provide.

This 2,800+ word guide will cover everything you need to know about:

• What HAR files contain and how browsers generate them
• Step-by-step setup in over 8 major browsers
• Comparing 10+ popular HAR analysis tools
• Methodology to analyze metrics and waterfalls
• Real-world web debugging case studies
• Potential privacy risks and sanitization

And more…

By the end, you‘ll be equipped with expert techniques to leverage these HTTP request/response archives to diagnose some of the most complex issues web applications can run into.

What Exactly is a HTTP Archive (HAR) File?

First, some background on what the term HTTP archive file refers to.

A HTTP archive file or .har file essentially captures a detailed log of all network requests made by the browser when loading a web page, as well as the corresponding responses returned from servers.

This reporting enables drilling down into granular timing data, header fields, and status codes for these back-and-forth interactions that happen invisible to the user while visiting a website.

Standardized under the HAR 1.2 specification, this JSON formatted file contains sections capturing:

1. Metadata – Info like HAR version, when file was created, URL of page visited

2. Browser Details – User agent string, cookies enabled, browser window size etc

3. Entries – An array of HTTP request-response pairs logged

Each entry contains extremely detailed data like:

• Request URL, method, headers, postData
• Response status code, status text, headers, content size
• Exact start time & time taken for response in milliseconds
• And more

This raw information is hard for humans to parse manually – which is where HAR analyzer tools come in handy.

But understanding the structure helps grasp the breadth of diagnostics data contained within.

Let‘s next see how we can actually generate these network activity reports across all major browsers.

Step-by-Step Guide to Generating HAR Files from Browsers

While most modern browsers include developer tools to export network activity logs, the exact menus and options vary slightly.

Here is an overview of HAR file generation across 8 major web browsers – Chrome, Firefox, Edge, Safari, Opera, Brave, Vivaldi and Internet Explorer.

| Browser             | Steps to Generate HAR File                                              |
| -----------------  | ------------------------------------------------------------------       |                                   
| Google Chrome       | Open DevTools > Network tab > Record > Export HAR                        |   
| Mozilla Firefox     | Open DevTools > Network tab > Check Persist Logs > Save All as HAR     |
| Microsoft Edge      | DevTools > Network > Record > Export HAR                                 |
| Apple Safari        | Develop > Show Web Inspector > Network > Preserve Log > Save            |   
| Opera               | Advanced > Developer Tools > Network > Logging On > Export Requests     |
| Brave               | Open DevTools > Network > Record > Export HAR                            |
| Vivaldi             | View > Developer Tools > Network > Record > Save all as HAR             |   
| Internet Explorer 11| Open F12 Tools > Network > Record > Export HAR                            |

Now let‘s go through 4 of the most commonly used browsers in more depth.

Google Chrome HAR Capture Process

Chrome DevTools provide the most refined interface for generating detailed HTTP archives of network activity.

Step 1) Navigate to the page causing performance issues in a new tab

Step 2) Open Developer Tools using Ctrl+Shift+I (Windows/Linux) or Cmd+Opt+I (Mac)

Step 3) Go to the Network tab

Step 4) Check box is selected for "Preserve Log"

This ensures complete recording history is maintained

Step 5) Click the record button (round red button) to start capturing traffic

Step 6) Reload the page and go through steps to reproduce the problems experienced

This will log all associated network requests

Step 7) When ready, click "Export HAR" button to save log files to disk

The result is a complete HAR file containing granular details of all requests and responses while browsing that page.

Mozilla Firefox HAR Capture

Step 1) Load the web page causing performance complaints

Step 2) Open Developer Tools via Ctrl+Shift+I (Windows/Linux) or Cmd+Opt+I (Mac)

Step 3) Switch to the Network tab

Step 4) Check box to enable "Persist Logs" option at bottom

This ensures complete history is saved

Step 5) Click on round Record button to start recording all activity

Step 6) Refresh page and reproduce any slow loading or errors

Step 7) Right click on any entry in left request list pane

Step 8) Choose "Save All As HAR" from context menu

This exports the entire logged network activity as a .har file for analysis.

Microsoft Edge HAR Capture

Step 1) Load the problematic web page

Step 2) Open Developer Tools using Ctrl+Shift+I keyboard shortcut

Step 3) Go to Network tab

Step 4) Check "Preserve log" option is enabled

Step 5) Click on round Record button to start recording

Step 6) Refresh page and repeat steps triggering issue

Step 7) Click on Export HAR button

Step 8) Save exported file to disk

Review in analyzer tools to investigate issues!

Apple Safari Web Inspector HAR Capture

Step 1) In Safari browser, enable Develop menu via Preferences > Advanced > Show Develop menu

Step 2) Load problematic page

Step 3) From menu, choose Develop > Show Web Inspector

Step 4) Select Network tab

Step 5) Check "Preserve Log" option at bottom

Step 6) Click Record button to start recording all activity

Step 7) Reload the page and reproduce any slowness/errors

Step 8) Right click any request entry and choose Save to export logs

Now you have a detailed HAR file capturing all network requests made during that browsing session.

As you can see, all browsers provide options to record network interactions and export HTTP archives. The process follows the same general flow – record activity, recreate issues, and export to HAR format.

Next let‘s explore specialized tools that give us the best visualization and analysis capabilities on these log files.

HAR File Viewers & Analysis Tools Compared

While HAR files can be opened in any text editor, dedicated analysis tools and web viewers provide far better ways to make sense of the dense data they contain in a developer-friendly graphical format.

Here is an overview of 10 popular HAR analysis tools:

| Tool                       | Platform    | Key Features                                                                 |
| -------------------------- | -------------| ------------------------------------------------------------------------------|  
| Google HAR Analyzer        | Web app       | Intuitive waterfall charts, filters, summary reports                            |
| HAR Viewer by Softwareishard | Web app      | Feature-packed analysis with requests table, graphs, alerts                    |  
| Chrome DevTools           | Chrome extension | Tight browser integration, network requests inspection                        |
| HTTP Toolkit             | Desktop app | High-performance analysis, modification capabilities                          |
| HAR.tech                 | Web app       | Automated performance checks and suggestions                              |
| HAR Tabs                  | Chrome extension | Quickly visualize page timings and failures                                     |  
| HARalyzer                 | Web app       | Python-based analysis through notebooks and REST API access                   |
| SpeedCurve                 | Web app       | Compare HARs visually, page load heatmaps                                     |  
| Pingman Tools             | Desktop app | Comprehensive HTTP client testing and API access                                |
| SkyHAR                    | Desktop app | Native macOS viewer with detailed timings table                          |

Let‘s analyze some of the stand-out options:

Google HAR Analyzer – Simple browser-based tool from Google for quick visualization of HAR file contents. Handy graphs show page loading waterfall with color coded requests. Easy filtering by text search or request type.

HAR Viewer – One of the most full-featured web apps for parsing HAR logs via an intuitive user interface. tons of analysis options from detailed tables, color codes, import, alerting and automated checks.

Chrome DevTools – Tight developer tools integration where imported HAR files can be analyzed leveraging all the existing Network tab features. Fast and responsive for large files. Requires Chrome browser.

HTTP Toolkit – Desktop app providing high-performance analysis capabilities. Advanced features like modifying requests, intercepting traffic, mocking endpoints. Free and paid versions available.

The best option depends on use case – either quick sharing with stakeholders, developer debugging or scripting analysis across tests.

Next let‘s go over how to process HAR file contents to extract performance insights.

10 Steps to Effectively Analyze HTTP Archive Files

Here is a methodology to leverage HAR visualizer tools effectively for debugging web app issues:

Step 1) For slow page load complaints, capture HAR file of affected page

Step 2) Compare with HAR from normal load to identify outlier requests

Step 3) Filter to specific type – CSS, JS, XHR, Media etc

Step 4) Sort requests by longest duration to find slowest

Step 5) Check if static assets have appropriate caching headers

Step 6) Spot error status codes breaking page functionality

Step 7) Analyze network waterfall for long gaps indicating delays

Step 8) Document exact request URL, headers, response codes

Step 9) Verify no sensitive data like tokens are exposed

Step 10) Share sanitized HAR file with web developers to address underlying cause

Let‘s break this down into more detail with an example…

Real-World Web Performance Issue & HAR Analysis

A user reported videos were failing to load properly on a video streaming site. The title cards would load but playback failed frequently.

Step 1 – Generate HAR File during video playback attempts to log all associated requests

Step 2 – Filter Requests to isolate video related assets only – filter to .mp4, .ts extensions

Step 3 – Inspect Video Entries and identify continuously changing dynamic URLs with no caching

Step 4 – Check Status Codes finding 500 errors when requesting the dynamic video URLs

Step 5 – Review Waterfall showing long gaps between video URL requests and error responses

Step 6 – Document the failed dynamic request patterns and exact codes

Step 7 – Redact any session tokens before sharing HAR file excerpt

Step 8 – Share with site developers who identify the backend video prep service failing under load

This demonstrates how detailed review of granular information within HAR files helped root cause hard to diagnose playback failures.

Let‘s run through a few more real-world examples where HAR analysis provided the breakthrough…

#1: Identifying Uncached Static Assets

Site owners noticed consistent performance complaints around slow load times. HAR analysis revealed the culprit – all static image and CSS assets were missing cache headers.

Every user request resulted in redundant and slow trips back to origin servers instead of using cached copies.

Adding Cache-Control headers for hashes/versioned filenames enabled 1+ second load time improvements!

#2: Diagnosing Redirect Chains

In another case, pages sporadically loaded slowly for some users. HTTP archive analysis showed long chains of unnecessary 302 redirects before final content got served.

This was likely due to bad link canonicalizations or remnants from outdated URLs. Eliminating redundant hops provided faster response.

#3: Catching Client-Side Bottlenecks

Web developers suspected slowness complaints were arising from backend API latency. But HAR analysis proved otherwise.

The API calls were fast but the browser took extremely long to process JavaScript after API responses returned. The frontend code was doing too much work blocking the thread.

Optimizing client-side JavaScript parsing and using Web Workers resolved this perception of lag without changing backend code!

This demonstrates just some of the ways expert analysis into HTTP traffic via HARs can provide vital diagnostics data and breakthrough insights no other method can match.

Now that we‘ve covered the essentials of generating and analyzing HAR files, let‘s briefly discuss some privacy considerations when dealing with such raw network logs.

Privacy Risks in HAR Files and Sanitization Tips

While HTTP archive analysis provides immense benefits, collecting and potentially sharing such detailed traffic logs also introduces privacy risks to consider.

Some ways sensitive user data can end up exposed in HAR files:

• Session cookies, tokens in headers
• Variables in GET/POST request parameters
• Personal identifiers in URLs
• Passwords included in Authorization creds

Much of this is is transmitted encrypted during transit but archived unencrypted in HAR files.

Before sharing your HTTP logs externally with any site owners or public issue trackers, manually inspect and sanitize as needed.

Common sanitization tips:

• Visually review all request headers/parameters
• Mask any tokens using placeholder values like *****
• Search for instances of email/usernames
• Remove cookies if not required to reproduce issues
• Scrub POST data fields irrelevant for diagnostics
• Validate no plaintext passwords leaked

With some caution, it is possible to extract only relevant subsets of HTTP data in shareable manner for collaborative troubleshooting while still respecting user privacy.

Conclusion & Next Steps

Analyzing HTTP traffic through archived HTTP logs provides invaluable diagnostics capability to developers struggling with complex performance and functional issues in modern web applications.

We walked through:

• The breadth of granular metadata captured in HAR files spanning browser details, timing, headers and content
• Step-by-step setup to export HAR files across 8 major browsers
• Comparison of 10+ popular HAR analysis tools evaluating capabilities
• Real-world examples where expert HAR analysis provided breakthrough insights
• Methodology for effective analysis by examining key metrics
• Privacy considerations when recording/sharing network activity logs

As web applications grow more complex with single-page apps, dynamic client-side rendering, heavy media assets and real-time updates – it becomes infinitely harder to debug problems users encounter.

Analyzing HTTP Archive files helps cut through the ambiguity to extract definitive proof of what occurred during a user‘s browsing session at a near microscopic level.

Using the tools and techniques covered here, you‘ll be well equipped to leverage HAR diagnostics capabilities for your own sites. Combine with other approaches like real user monitoring and client-side logging to build comprehensive visibility.

Hopefully this guide has provided a complete overview to help demystify HAR files – let us know if you have any other questions!

Author: Neil Lasrado

Neil is a senior web performance engineer and technical writer based in Mumbai, India with 5+ years expertise in web application monitoring, optimization and troubleshooting.