Dot-Net

如何即時壓縮http請求而不在記憶體中載入壓縮緩衝區

  • March 11, 2019

我需要在 http post 請求中將大量數據發送到支持 gzip 編碼請求的伺服器。

從一個簡單的開始

public async Task<string> DoPost(HttpContent content)
{
 HttpClient client = new HttpClient();
 HttpResponseMessage response = await client.PostAsync("http://myUri", content);

 response.EnsureSuccessStatusCode();
 return await response.Content.ReadAsStringAsync();
}

我剛剛添加了預壓縮

public async Task<string> DoPost(HttpContent content, bool compress)
{
 if (compress) 
   content= await CompressAsync(content);

 return await DoPost(content);
}

private static async Task<StreamContent> CompressAsync(HttpContent content)
{
 MemoryStream ms = new MemoryStream();
 using (GZipStream gzipStream = new GZipStream(ms, CompressionMode.Compress, true))
 {
   await content.CopyToAsync(gzipStream);
   await gzipStream.FlushAsync();
 }

 ms.Position = 0;
 StreamContent compressedStreamContent = new StreamContent(ms);
 compressedStreamContent.Headers.ContentType = content.Headers.ContentType;
 compressedStreamContent.Headers.Add("Content-Encoding", "gzip");

 return compressedStreamContent;
}

它工作得很好,但壓縮數據在發送請求之前完全載入到記憶體中。我希望能夠在以流方式發送期間動態壓縮數據。

為此,我嘗試了以下程式碼:

private static async Task<HttpContent> CompressAsync2(HttpContent content)
{
 PushStreamContent pushStreamContent = new PushStreamContent(async (stream, content2, transport) =>
 {
   using (GZipStream gzipStream = new GZipStream(stream, CompressionMode.Compress, true))
   {
     try
     {
       await content.CopyToAsync(gzipStream);
       await gzipStream.FlushAsync();
     }
     catch (Exception exception)
     {
       throw;
     }
   }
 });
 pushStreamContent.Headers.ContentType = content.Headers.ContentType;
 pushStreamContent.Headers.Add("Content-Encoding", "gzip");

 return pushStreamContent;
}

但它永遠不會超出 CopyToAsync(gzipStream)。FlushAsync 永遠不會執行,也不會引發異常,Fiddler 也看不到任何文章開始。

我的問題是:

  • 為什麼 CompressAsync2 不起作用?
  • 如何在發送期間即時壓縮而不在記憶體中載入壓縮緩衝區?

任何幫助將不勝感激。

嘗試使用 WebAPIContrib 中的 CompressedContent 類https://github.com/WebApiContrib/WebAPIContrib/blob/master/src/WebApiContrib/Content/CompressedContent.cs

public async Task<string> DoPost(HttpContent content)
{
 HttpClient client = new HttpClient();
 HttpResponseMessage response = await client.PostAsync("http://myUri", 
                                new CompressedContent(content,"gzip"));

 response.EnsureSuccessStatusCode();
 return await response.Content.ReadAsStringAsync();
}

PS這只會流式傳輸.net 4.5上的內容。.net 4 版本的 HttpWebRequest 總是緩衝發送的內容。

PPS 為每個請求創建一個新的 HttpClient 並不是使用 HttpClient 的最佳方式。這樣做將強制為每個請求創建一個新的 TCP 連接。

引用自:https://stackoverflow.com/questions/16673714