Streaming vs Polling: Azure Durable Functions for Large SAP OData Payloads

The Problem: 70MB SAP Payloads in Memory

A common pattern when fetching SAP product data via OData is:

• Call HttpClient.GetAsync(url)
• Await the full response body: response.Content.ReadAsStringAsync()
• Deserialise with JsonConvert.DeserializeObject<T>(json)

For small payloads this works fine. For SAP product catalogues with 50,000+ items, each response can exceed 70MB. Loading this entirely into a string allocates 70MB on the heap per call — and if you have 4 product categories running in parallel via Durable Functions fan-out, you're looking at 280MB+ of heap pressure per execution cycle.

The result: OutOfMemoryException, failed runs, and data gaps in Cosmos DB.

The Solution: ResponseHeadersRead + JsonTextReader

The fix is to stream the response instead of buffering it. Two key changes:

1. Use HttpCompletionOption.ResponseHeadersRead

Instead of GetAsync(url) which buffers the full body, use:

• GetAsync(url, HttpCompletionOption.ResponseHeadersRead)
• This returns as soon as response headers arrive — before the body is downloaded
• You then get a stream: await response.Content.ReadAsStreamAsync()

2. Stream-deserialise with JsonTextReader

Use Newtonsoft Json.NET's JsonTextReader to deserialise directly from the stream:

• Objects are yielded one at a time as the stream arrives
• Peak memory usage stays roughly constant regardless of payload size
• Each item can be written to Cosmos DB as it is deserialised — true streaming pipeline

Durable Functions Architecture

The overall orchestration pattern:

• Orchestrator function — fans out one activity per product category
• Activity function — streams OData response and upserts to Cosmos DB in batches of 100
• Retry policy — RetryOptions with 3 attempts, exponential backoff
• HttpClient — injected via IHttpClientFactory, not newed up per call
• Key Vault — SAP credentials via DefaultAzureCredential, not appsettings

OData Filter Encoding

A frequent source of empty responses is incorrect OData filter encoding. The $filter parameter must be URL-encoded correctly — spaces as %20, single quotes escaped as %27. Validate your filter strings against the SAP Gateway test endpoint before deploying.

Results

After implementing streaming: zero out-of-memory failures, 99.97% successful job completion, peak memory reduced from 280MB+ to under 40MB per execution, and processing time reduced by approximately 35%.