Web Requests

Sure — I’ve rewritten your notes into a clean, structured, and professional explanation suitable for a website or learning page. I kept the technical accuracy but improved clarity, grammar, flow, and consistency.

Web Requests & Networking Fundamentals

Web Request Tools

1. cURL

Command-line tool used to send HTTP requests.

Key points:

  • cURL does not render HTML, JavaScript, or CSS — it only retrieves raw responses
  • Download a file:
    1
    2
    curl -O http://example.com/index.html
     
  • Silent mode:
    1
    2
    curl -s -O http://example.com/index.html
     
  • View help:
    1
    2
    3
    curl -h
    man curl
     

Example:

1
2
curl http://192.123.55.114:80/page.php
 

2. Browser Developer Tools

Shortcuts:

  • Open DevTools: Ctrl + Shift + I
  • Network tab: Ctrl + Shift + E
  • Console tab: Ctrl + Shift + K

The Network tab allows you to inspect all HTTP requests and responses made by a web page.

HTTP & HTTPS Basics

URL Structure

A URL requires:

  • Scheme (http or https)
  • Host (domain or IP address)

Example:

1
2
https://www.example.com/page?param=value
 

DNS Resolution

When accessing a domain:

  1. The browser first checks /etc/hosts
  2. If not found, it queries a DNS server

HTTP vs HTTPS

HTTP

  • Data is transmitted in clear text
  • Vulnerable to Man-in-the-Middle (MITM) attacks
  • Credentials can be intercepted

HTTPS

  • Encrypts all traffic
  • Protects against interception and tampering

Typical HTTPS handshake:

  1. Client Hello
  2. Server Hello
  3. Certificate exchange
  4. Key handshake
  5. Encrypted communication

⚠ Using curl -k skips certificate validation and exposes you to MITM attacks.

HTTP Requests & Responses

Structure

An HTTP response contains:

  • Headers
  • Body

Separated by a blank line.

View full request/response:

1
2
curl -v https://example.com
 

Headers only:

1
2
curl -I https://example.com
 

Common HTTP Headers

General

  • Date
  • Connection

Entity

  • Content-Type
  • Content-Length
  • Content-Encoding

Request

  • Host
  • User-Agent
  • Referer
  • Accept
  • Cookie
  • Authorization

Response

  • Server
  • Set-Cookie
  • WWW-Authenticate

Security

  • Content-Security-Policy
  • Strict-Transport-Security
  • Referrer-Policy

HTTP Methods

Method Purpose
GET Retrieve data
POST Send data
HEAD Headers only
PUT Create/update resource
DELETE Remove resource
PATCH Partial update
OPTIONS Server capabilities

Most web apps use GET and POST.
REST APIs commonly use PUT and DELETE.

Authentication with cURL

Basic Auth

1
2
curl -u username:password http://server-ip/
 

Or embedded:

1
2
curl http://username:password@server-ip/page.php
 

POST Requests

POST sends data in the request body instead of the URL.

Benefits:

  • Less logging
  • No URL length limit
  • Cleaner encoding

Example:

1
2
curl -X POST -d "login=user&password=pass" http://example.com
 

Follow redirects:

1
2
curl -L ...
 

Cookies & Sessions

After authentication, servers often return a session cookie:

1
2
Set-Cookie: PHPSESSID=abc123
 

Reuse it:

1
2
curl -b "PHPSESSID=abc123" http://server-ip/
 

Or:

1
2
curl -H "Cookie: PHPSESSID=abc123" http://server-ip/
 

Possessing a valid cookie may be enough to access authenticated content.

JSON & APIs

Specify JSON content:

1
2
-H "Content-Type: application/json"
 

CRUD mapping:

Operation HTTP Method
Create POST
Read GET
Update PUT
Delete DELETE

Example API request:

1
2
curl -s http://server-ip/api.php/city/london | jq
 

Networking Fundamentals

Network Types

  • WAN – Internet-scale networks
  • LAN – Internal home or office networks
  • WLAN – Wireless LAN (Wi-Fi)
  • VPN – Secure private networking
    • Site-to-site
    • Remote access
    • SSL VPN
  • MAN – Regional networks
  • GAN – Global internet backbone
  • WPAN – Bluetooth & personal networks

Network Topologies

  • Point-to-point
  • Star
  • Mesh
  • Bus
  • Ring
  • Tree
  • Hybrid
  • Daisy chain

Proxies

  • Forward Proxy – Filters outgoing traffic
  • Reverse Proxy – Filters incoming traffic
  • Transparent Proxy – Intercepts traffic without client configuration

VPNs are not proxies.

Networking Models

OSI (7 layers)

Physical → Data Link → Network → Transport → Session → Presentation → Application

TCP/IP (4 layers)

Link → Internet → Transport → Application

IP Addressing

IPv4

  • 4 octets (0–255)
  • Example: 192.168.1.1
  • Divided into:
    • Network portion
    • Host portion

CIDR

Example:

1
2
192.168.10.39/24
 

  • /24 = subnet mask bits
  • Remaining bits = hosts

MAC Addresses

  • 48-bit hardware address
  • Hexadecimal format
  • First 3 bytes = manufacturer (OUI)
  • Last 3 bytes = device ID

Used for local network communication.

ARP (Address Resolution Protocol)

Maps IP addresses to MAC addresses when communicating within a subnet.

Browser Fingerprinting (HTTP Footprint)

User identification can occur without cookies using:

Collected via HTTP Headers

  • User-Agent
  • Language
  • Encoding
  • Platform

Collected via JavaScript & HTML5

  • Screen resolution
  • Timezone
  • Hardware concurrency
  • Installed fonts
  • Canvas fingerprint
  • WebGL vendor/renderer
  • Audio context
  • Keyboard layout
  • Battery status
  • Network connection

Cookies & Tracking

  • _ga (Google Analytics)
    • Unique per website + user
    • Expires ~2 years
  • _gid
    • Short-lived identifier

Combined with IP and browser fingerprinting, these can uniquely identify users.

Security Note on Certificates

Failing to verify TLS certificates exposes applications to:

  • Man-in-the-middle attacks
  • Data interception
  • Credential theft

Always validate certificates in production environments.