Dot-Net

如何將 CsvHelper 記錄添加到 DataTable 以用於 SqlBulkCopy 到數據庫

  • March 30, 2022

我正在嘗試使用 CsvHelper 讀取 CSV 文件,將每條記錄載入到 DataTable 中,然後使用 SqlBulkCopy 將數據插入數據庫表中。使用目前程式碼,在向 DataTable 添加行時出現異常。例外情況是:“無法將 ‘MvcStockAnalysis.Models.StockPrice’ 類型的對象轉換為 ‘System.IConvertible’ 類型。無法儲存在日期列中。預期類型為 DateTime。”

範例 CSV 文件來自 yahoo Finance。例如: http: //ichart.yahoo.com/table.csv ?s=MMM&a=0&b=1&c=2010&d=0&e=17&f=2014&g=d&ignore=.csv

CSV 文件包含以下標題: Date Open High Low Close Volume Adj Close

我正在將 CSV 文件讀入的模型:

namespace MvcStockAnalysis.Models
{
   using System;
   using System.Collections.Generic;

   public partial class StockPrice
   {
       public int Id { get; set; }
       public System.DateTime Date { get; set; }
       public int CompanyId { get; set; }
       public double High { get; set; }
       public double Low { get; set; }
       public double Close { get; set; }
       public double AdjClose { get; set; }
       public double Open { get; set; }
       public double Volume { get; set; }

       public virtual Company Company { get; set; }
   }
}

CSV 文件到 StockPrice 類的映射使用以下內容:

public class StockPriceClassMap : CsvClassMap<StockPrice>
{
   public override void CreateMap()
   {
       Map(m => m.Date).Name("Date");
       Map(m => m.Close).Name("Close");
       Map(m => m.AdjClose).Name("Adj Close");
       Map(m => m.High).Name("High");
       Map(m => m.Low).Name("Low");
       Map(m => m.Open).Name("Open");
       Map(m => m.Volume).Name("Volume");
   }
}

嘗試將 CsvHelper 記錄添加到 DataTable 的程式碼如下:

var connectionstring = ConfigurationManager.ConnectionStrings["MvcStockAnalysis.Models.MvcStockAnalysisContext"];
var connection = new SqlConnection();
connection.ConnectionString = connectionstring.ToString();
var destinationTableName = "StockPrices";
var company = db.Company
           .Where(c => c.Symbol == "MMM")
           .FirstOrDefault();

try
{
   string path = HttpContext.Server.MapPath("~/App_Data/" + company.Symbol + @".csv");

   if (System.IO.File.Exists(path))
   {     

       using (StreamReader sr = new StreamReader(path))
       {
           using (var csv = new CsvReader(sr))
           {
               DataTable dt = new DataTable("StockPrices");
               csv.Configuration.HasHeaderRecord = true;
               csv.Configuration.RegisterClassMap<StockPriceClassMap>();

               dt.Columns.Add(new DataColumn("Date", typeof(DateTime)));
               dt.Columns.Add(new DataColumn("Close", typeof(Double)));
               dt.Columns.Add(new DataColumn("AdjClose", typeof(Double)));
               dt.Columns.Add(new DataColumn("High", typeof(Double)));
               dt.Columns.Add(new DataColumn("Low", typeof(Double)));
               dt.Columns.Add(new DataColumn("Open", typeof(Double)));
               dt.Columns.Add(new DataColumn("Volume", typeof(Double)));
               dt.Columns.Add(new DataColumn("CompanyId", typeof(Double)));
               var records = csv.GetRecords<StockPrice>().ToList();
               foreach (var record in records)
               {                                    
                   record.CompanyId = company.Id;
                   dt.Rows.Add(record);
               }
               // add dt to the database
               using (var bulkCopy = new SqlBulkCopy(connection.ConnectionString))
               {
                   // DataTable column names match my SQL Column names, so I simply made this loop. 
                   foreach (DataColumn col in dt.Columns)
                   {
                       bulkCopy.ColumnMappings.Add(col.ColumnName, col.ColumnName);
                   }
                   bulkCopy.DestinationTableName = destinationTableName;
                   bulkCopy.WriteToServer(dt);
               }
           }
       }
   }
   connection.Close();
}
catch (Exception e)
{
   Console.Write(e.Message);
}

如何將 CsvHelper 記錄添加到 DataTable 以用於 SqlBulkCopy 到數據庫?

如果我沒記錯的話,你應該可以用更少的程式碼來做到這一點。在進入任何一個之前,您都不必進入另一個班級DataTable

while( csv.Read() )
{
   var row = dt.NewRow();
   foreach( DataColumn column in dt.Columns )
   {
       row[column.ColumnName] = csv.GetField( column.DataType, column.ColumnName );
   }
   dt.Rows.Add( row );
}

Josh 去年增加了對讀取標題的支持,以下塊可能對那些只想使用 CSV 文件架構建構 DataTable 的人有用。我想將此作為對 Josh 答案的評論發布,因為它只是一個小的修改,但作為答案發布,因為我無法在評論中格式化程式碼塊。

   private DataTable BuildDataTable()
   {
       var dt = new DataTable();
       using (var textReader = new StreamReader(_path))
       {
           using (var csv = new CsvReader(textReader))
           {
               csv.Read();
               csv.ReadHeader();
               foreach (var header in csv.FieldHeaders)
               {
                   dt.Columns.Add(header);
               }
               while (csv.Read())
               {
                   var row = dt.NewRow();
                   foreach (DataColumn column in dt.Columns)
                   {
                       row[column.ColumnName] = csv.GetField(column.DataType, column.ColumnName);
                   }
                   dt.Rows.Add(row);
               }
           }
       }
       return dt;
   }

引用自:https://stackoverflow.com/questions/21257827