HTTP/1.1¶
palfrey.protocols.http
¶
HTTP/1.1 request parsing and response encoding with multiple backend support.
This module implements HTTP/1.1 protocol handling including request parsing via httptools (C-based, preferred) or h11 (pure Python fallback), ASGI scope construction, request body streaming, and response encoding with keep-alive decision logic. The module provides building blocks for keep-alive detection, 100-continue handling, chunked transfer encoding, and content-length calculations required for standards-compliant HTTP/1.1 message framing.
Key Design Decisions: - Dual-backend parser selection: httptools by default (fast C library) falls back to h11 for compatibility or when C extensions are unavailable. - Request/Response dataclasses normalize heterogeneous wire formats into consistent, Python-friendly structures (bytes for headers, chunks for body). - Keep-alive is determined by HTTP version and Connection header, not by response body completion—allowing pipelined requests to be queued immediately. - 100-continue handling is async-callback-driven for proper backpressure integration.
Key Functions
- build_http_scope: Constructs ASGI scope dict from HTTPRequest and metadata.
- run_http_asgi: Main coroutine cycling through request/response pairs.
- encode_http_response: Serializes HTTPResponse to wire bytes with chunking.
- read_http_request: Parses wire bytes into HTTPRequest dataclass.
- should_keep_alive: Determines if connection should remain open after response.
HTTPRequest
dataclass
¶
Data container for a parsed HTTP request, including headers and body content.
This class normalizes the body representation into both a contiguous byte string and a list of chunks, facilitating easier processing for both synchronous logic and asynchronous streaming.
| ATTRIBUTE | DESCRIPTION |
|---|---|
method |
The HTTP method (e.g., 'GET', 'POST').
TYPE:
|
target |
The full request URI or path.
TYPE:
|
http_version |
The protocol version (e.g., 'HTTP/1.1').
TYPE:
|
headers |
Header pairs.
TYPE:
|
body |
The complete request body as a single byte string.
TYPE:
|
body_chunks |
The body split into chunks, as received from the wire.
TYPE:
|
Source code in palfrey/protocols/http.py
__post_init__()
¶
Synchronizes the body and body_chunks attributes after initialization.
Ensures that if body_chunks is provided, the body attribute reflects the concatenated content. If only body is provided, body_chunks is populated with a single-element list containing that body.
Source code in palfrey/protocols/http.py
HTTPResponse
dataclass
¶
Representation of an outgoing HTTP response constructed from ASGI events.
This object acts as a collector for headers and body chunks sent by the application before they are serialized to the network socket.
| ATTRIBUTE | DESCRIPTION |
|---|---|
status |
HTTP status code (default 500).
TYPE:
|
headers |
List of header tuples in bytes.
TYPE:
|
body_chunks |
Fragments of the response body.
TYPE:
|
chunked_encoding |
Whether the response uses Transfer-Encoding: chunked.
TYPE:
|
suppress_body |
If True, the body is omitted (e.g., for HEAD requests).
TYPE:
|
Source code in palfrey/protocols/http.py
read_http_request(reader, *, max_head_size=1048576, body_limit=4194304, parser_mode='auto')
async
¶
Reads and parses a full HTTP request (head and body) from the reader.
| PARAMETER | DESCRIPTION |
|---|---|
reader
|
The client stream reader.
TYPE:
|
max_head_size
|
Max allowed bytes for the request line and headers.
TYPE:
|
body_limit
|
Max allowed bytes for the body.
TYPE:
|
parser_mode
|
Choice of parser ('httptools', 'h11', or 'auto').
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
HTTPRequest | None
|
HTTPRequest | None: The parsed request, or None if the client disconnected. |
Source code in palfrey/protocols/http.py
build_http_scope(request, *, client, server, root_path, is_tls)
¶
Converts an internal HTTPRequest into an ASGI 3.0 scope dictionary.
| PARAMETER | DESCRIPTION |
|---|---|
request
|
Parsed request data.
TYPE:
|
client
|
IP and port of the client.
TYPE:
|
server
|
IP and port of the server.
TYPE:
|
root_path
|
The mounting point of the application.
TYPE:
|
is_tls
|
True if connection is encrypted.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Scope
|
A dictionary conforming to the ASGI HTTP specification.
TYPE:
|
Source code in palfrey/protocols/http.py
run_http_asgi(app, scope, request_body, *, expect_100_continue=False, on_100_continue=None, on_response_start=None, on_response_body=None)
async
¶
Orchestrates the ASGI request/response lifecycle with high-performance formatting.
This function manages the strict state machine required by the ASGI HTTP specification. It incorporates several critical performance optimizations:
- Single-Chunk Optimization: Defers finalizing headers until the first body chunk arrives.
If the app yields a single chunk (
more_body=False) without aContent-Length, it calculates it instantly to avoid the overhead of HTTP chunked transfer framing. - In-Flight Chunk Framing: If
Transfer-Encoding: chunkedis actually required, it formats the hex framing continuously during the app'ssend()loop rather than post-processing.
| PARAMETER | DESCRIPTION |
|---|---|
app
|
The user-provided ASGI application callable.
TYPE:
|
scope
|
The ASGI connection scope dictionary.
TYPE:
|
request_body
|
The pre-buffered input body chunks.
TYPE:
|
expect_100_continue
|
If the client expects a 100-Continue response.
TYPE:
|
on_100_continue
|
Callback to trigger the 100 status code transmission.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
HTTPResponse
|
The resulting parsed response state ready for wire encoding.
TYPE:
|
| RAISES | DESCRIPTION |
|---|---|
RuntimeError
|
If the ASGI application violates the protocol sequence or crashes. |
Source code in palfrey/protocols/http.py
630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 | |
append_default_response_headers(response, config, *, default_headers=None)
¶
Applies configured default headers to the response.
Injects headers like 'Server' and 'Date' if enabled and not already present.
| PARAMETER | DESCRIPTION |
|---|---|
response
|
The response object to update.
TYPE:
|
config
|
Application configuration.
TYPE:
|
default_headers
|
Cached list of headers for fast-path insertion.
TYPE:
|
Source code in palfrey/protocols/http.py
encode_http_response_head(response, keep_alive)
¶
Serializes the HTTP response status line and headers.
Handles status line and headers while preserving chunked transfer metadata for callers that stream body chunks separately.
| PARAMETER | DESCRIPTION |
|---|---|
response
|
The response to serialize.
TYPE:
|
keep_alive
|
Whether the connection should be kept alive.
TYPE:
|
| YIELDS | DESCRIPTION |
|---|---|
bytes
|
Serialized status and header fragments.
TYPE::
|
Source code in palfrey/protocols/http.py
encode_http_response_body_chunk(response, body, *, more_body)
¶
Serializes a single ASGI response body event into wire bytes.
Source code in palfrey/protocols/http.py
encode_http_response_chunks(response, keep_alive)
¶
Serializes the HTTPResponse into a sequence of bytes chunks.
Handles status line, headers, and body encoding including chunked transfer.
| PARAMETER | DESCRIPTION |
|---|---|
response
|
The response to serialize.
TYPE:
|
keep_alive
|
Whether the connection should be kept alive.
TYPE:
|
| YIELDS | DESCRIPTION |
|---|---|
bytes
|
Serialized fragments of the HTTP response.
TYPE::
|
Source code in palfrey/protocols/http.py
encode_http_response(response, keep_alive)
¶
Zero-copy serialization of the HTTPResponse into raw wire bytes.
This function is aggressively optimized to avoid intermediate string formatting
and memory allocations. It appends pre-encoded bytes into a flat list and performs
a single C-level b"".join().
| PARAMETER | DESCRIPTION |
|---|---|
response
|
The finalized response data collected from the app.
TYPE:
|
keep_alive
|
Indicates if the connection will remain open.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bytes
|
The fully serialized HTTP response ready for
TYPE:
|
Source code in palfrey/protocols/http.py
should_keep_alive(request, response)
¶
Determines if the TCP connection should persist based on headers and protocol version.
Handles the nuances between HTTP/1.0 explicit keep-alive and HTTP/1.1 explicit close.
| PARAMETER | DESCRIPTION |
|---|---|
request
|
The parsed incoming request.
TYPE:
|
response
|
The finalized outgoing response.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bool
|
True if the connection should be kept alive, False to close.
TYPE:
|
Source code in palfrey/protocols/http.py
is_websocket_upgrade(request)
¶
requires_100_continue(request)
¶
Verifies if the client is waiting for a '100 Continue' response before sending the body.