手工从github存储库下载插件等duckdb官方合并后也可以用install orc from community直接安装C:\dwget https://github.com/alitrack/duckdb_orc/releases/download/v0.2.1/orc-windows_amd64.duckdb_extension orc-windows_amd64.duckdb_exte 100%[] 11.39M 7.13MB/s in 1.6s 2026-09-02 18:41:37 (7.13 MB/s) - orc-windows_amd64.duckdb_extension saved [11940886/11940886] C:\dwget https://github.com/alitrack/duckdb_orc/releases/download/v0.2.1/orc-linux_amd64.duckdb_extension orc-linux_amd64.duckdb_extens 100%[] 10.69M 7.56MB/s in 1.4s 2026-09-02 18:41:56 (7.56 MB/s) - orc-linux_amd64.duckdb_extension saved [11206414/11206414]DuckDB v2.0.0-alpha有些版本没有插件不建议使用。C:\dduckdb0831 DuckDB v2.0.0-alpha39651 (Cyanoptera) Enter .help for usage hints. memory D load tpch; IO Error: Extension C:\Users\lt\.duckdb\extensions\v2.0.0-alpha39651\windows_amd64\tpch.duckdb_extension not found. Extension tpch is an existing extension. Install it first using INSTALL tpch. memory D install tpch; HTTP Error: Failed to download extension tpch at URL http://extensions.duckdb.org/v2.0.0-alpha39651/windows_amd64/tpch.duckdb_extension.gz (HTTP 404) Extension tpch is an existing extension. For more info, visit https://duckdb.org/docs/current/extensions/troubleshooting?versionv2.0.0-alpha39651platformwindows_amd64extensiontpch memory D .exit建议使用正式版本如DuckDB v1.5.5。C:\dduckdb155 DuckDB v1.5.5 (Variegata) Enter .help for usage hints. memory D load tpch; IO Error: Extension C:\Users\lt\.duckdb\extensions\v1.5.5\windows_amd64\tpch.duckdb_extension not found. Extension tpch is an existing extension. Install it first using INSTALL tpch. memory D install tpch; memory D load tpch; memory D call dbgen(sf2); ┌─────────┐ │ Success │ │ boolean │ └─────────┘ 0 rows memory D .timer on memory D .tables ────────────────────────────────────────────── memory ─────────────────────────────────────────────── ─────────────────────────────────────────────── main ──────────────────────────────────────────────── ┌─────────────────────────┐┌───────────────────────┐┌─────────────────────────┐┌──────────────────────┐ │ lineitem ││ part ││ orders ││ customer │ │ ││ ││ ││ │ │ l_orderkey bigint ││ p_partkey bigint ││ o_orderkey bigint ││ c_custkey bigint │ │ l_partkey bigint ││ p_name varchar ││ o_custkey bigint ││ c_name varchar │ │ l_suppkey bigint ││ p_mfgr varchar ││ o_orderstatus varchar ││ c_address varchar │ │ l_linenumber bigint ││ p_brand varchar ││ o_totalprice decimal ││ c_nationkey integer │ │ l_quantity decimal ││ p_type varchar ││ o_orderdate date ││ c_phone varchar │ │ l_extendedprice decimal ││ p_size integer ││ o_orderpriority varchar ││ c_acctbal decimal │ │ l_discount decimal ││ p_container varchar ││ o_clerk varchar ││ c_mktsegment varchar │ │ l_tax decimal ││ p_retailprice decimal ││ o_shippriority integer ││ c_comment varchar │ │ l_returnflag varchar ││ p_comment varchar ││ o_comment varchar ││ │ │ l_linestatus varchar ││ ││ ││ 300000 rows │ │ l_shipdate date ││ 400000 rows ││ 3.00 million rows │└──────────────────────┘ │ l_commitdate date │└───────────────────────┘└─────────────────────────┘ │ l_receiptdate date │ │ l_shipinstruct varchar │ │ l_shipmode varchar │ │ l_comment varchar │ │ │ │ 12.00 million rows │ └─────────────────────────┘ ┌─────────────────────┐┌───────────────────────┐┌─────────────────────┐┌─────────────────────┐ │ supplier ││ partsupp ││ nation ││ region │ │ ││ ││ ││ │ │ s_suppkey bigint ││ ps_partkey bigint ││ n_nationkey integer ││ r_regionkey integer │ │ s_name varchar ││ ps_suppkey bigint ││ n_name varchar ││ r_name varchar │ │ s_address varchar ││ ps_availqty bigint ││ n_regionkey integer ││ r_comment varchar │ │ s_nationkey integer ││ ps_supplycost decimal ││ n_comment varchar ││ │ │ s_phone varchar ││ ps_comment varchar ││ ││ 5 rows │ │ s_acctbal decimal ││ ││ 25 rows │└─────────────────────┘ │ s_comment varchar ││ 1.60 million rows │└─────────────────────┘ │ │└───────────────────────┘ │ 20000 rows │ └─────────────────────┘ memory D select sum(l_linenumber) from lineitem; ┌───────────────────┐ │ sum(l_linenumber) │ │ int128 │ ├───────────────────┤ │ 35994729 │ │ (35.99 million) │ └───────────────────┘ Run Time (s): real 0.011 user 0.015625 sys 0.000000 memory D copy (select l_linenumber from lineitem) to lll.parquet; Run Time (s): real 0.061 user 0.515625 sys 0.078125 memory D copy (select COLUMNS(* EXCLUDE l_comment) from lineitem) to lll.parquet; Run Time (s): real 1.788 user 7.234375 sys 10.890625先安装tpch插件生成8个表规模为2G先测试内置写入parquet文件。安装crc插件要注意如果已经装过旧版要force install。memory D install ./orc.duckdb_extension; Invalid Input Error: Installing extension orc failed. The extension is already installed but the origin is different. Currently installed extension is from repository http://community-extensions.duckdb.org, while the extension to be installed is from custom_path ./orc.duckdb_extension. To solve this rerun this command with FORCE INSTALL memory D force install ./orc.duckdb_extension; memory D load orc; memory D call dbgen(sf2); ┌─────────┐ │ Success │ │ boolean │ └─────────┘ 0 rows memory D .timer on memory D copy (select COLUMNS(* EXCLUDE l_comment) from lineitem) to lll.arc; Run Time (s): real 2.115 user 11.062500 sys 1.328125 memory D copy (select COLUMNS(* EXCLUDE l_comment) from lineitem) to lll.orc; Binder Error: write_orc: type Decimal128(38, 2) not supported by the ORC writer yet (supported: bool, int8-64, float32/64, date, timestamp, varchar, blob) Run Time (s): real 0.006 user 0.000000 sys 0.000000 memory D注意写错后缀arc默认写入csv文件。写对orc后缀后插件检测出不兼容的小数类型Decimal128报错退出。利用下面的脚本转换为支持的double类型再写入orc文件。 sql copy( select l_orderkey , l_partkey , l_suppkey , l_linenumber , l_quantity ::double l_quantity , l_extendedprice ::double l_extendedprice , l_discount ::double l_discount , l_tax ::double l_tax , l_returnflag , l_linestatus , l_shipdate , l_commitdate , l_receiptdate , l_shipinstruct , l_shipmode , from lineitem) to lll.orc;写入orc文件用时约是parquet格式的3倍。memory D copy( select l_orderkey , l_partkey , l_suppkey , l_linenumber , l_quantity ::double l_quantity , l_extendedprice ::double l_extendedprice , l_discount ::double l_discount , l_tax ::double l_tax , l_returnflag , l_linestatus , l_shipdate , l_commitdate , l_receiptdate , l_shipinstruct , l_shipmode , from lineitem) to lll.orc; Run Time (s): real 4.808 user 4.468750 sys 0.406250 memory D copy( select l_orderkey , l_partkey , l_suppkey , l_linenumber , l_quantity ::double l_quantity , l_extendedprice ::double l_extendedprice , l_discount ::double l_discount , l_tax ::double l_tax , l_returnflag , l_linestatus , l_shipdate , l_commitdate , l_receiptdate , l_shipinstruct , l_shipmode , from lineitem) to lll.parquet; Run Time (s): real 1.619 user 6.734375 sys 10.250000查询文件的时间差距更大相差30倍。注意orc插件写的文件列名被修改为了column_的形式这是DuckDB C API的限制。memory D select sum(l_linenumber) from lll.parquet; ┌───────────────────┐ │ sum(l_linenumber) │ │ int128 │ ├───────────────────┤ │ 35994729 │ │ (35.99 million) │ └───────────────────┘ Run Time (s): real 0.032 user 0.000000 sys 0.000000 memory D select sum(l_linenumber) from lll.orc; Binder Error: Referenced column l_linenumber not found in FROM clause! Candidate bindings: column_0, column_1, column_2, column_3, column_4 LINE 1: select sum(l_linenumber) from lll.orc; ^ Run Time (s): real 0.012 user 0.000000 sys 0.000000 memory D select sum(column_4) from lll.orc; ┌──────────────────┐ │ sum(column_4) │ │ double │ ├──────────────────┤ │ 305976330.0 │ │ (305.98 million) │ └──────────────────┘ Run Time (s): real 1.217 user 2.031250 sys 2.359375 memory D select sum(column_3) from lll.orc; ┌─────────────────┐ │ sum(column_3) │ │ int128 │ ├─────────────────┤ │ 35994729 │ │ (35.99 million) │ └─────────────────┘ Run Time (s): real 1.137 user 2.046875 sys 1.515625 memory D是否这种文件格式天生就慢不一定比如clickhouse两者的查询时间只差1倍可能是因为orc插件不支持并行。C:\dwsl rootDESKTOP-59T6U68:/mnt/c/d# ./clickhouse ClickHouse local version 26.8.1.1157 (official build). :) select sum(column_3) from file(lll.orc); SELECT sum(column_3) FROM file(lll.orc) Query id: 4426b39b-b538-48f3-a265-37e53449ae3d ┌─sum(column_3)─┐ 1. │ 35994729 │ -- 35.99 million └───────────────┘ 1 row in set. Elapsed: 0.112 sec. Processed 12.00 million rows, 254.23 MB (106.81 million rows/s., 2.26 GB/s.) Peak memory usage: 9.86 MiB. :) select sum(l_linenumber) from file(lll.parquet); SELECT sum(l_linenumber) FROM file(lll.parquet) Query id: fe9eef68-0d56-46d9-88e4-d1027a6bc423 ┌─sum(l_linenumber)─┐ 1. │ 35994729 │ -- 35.99 million └───────────────────┘ 1 row in set. Elapsed: 0.064 sec. Processed 12.00 million rows, 2.94 MB (187.68 million rows/s., 46.02 MB/s.) Peak memory usage: 47.16 MiB.
测试duckdb orc插件的写功能及读写性能
手工从github存储库下载插件等duckdb官方合并后也可以用install orc from community直接安装C:\dwget https://github.com/alitrack/duckdb_orc/releases/download/v0.2.1/orc-windows_amd64.duckdb_extension orc-windows_amd64.duckdb_exte 100%[] 11.39M 7.13MB/s in 1.6s 2026-09-02 18:41:37 (7.13 MB/s) - orc-windows_amd64.duckdb_extension saved [11940886/11940886] C:\dwget https://github.com/alitrack/duckdb_orc/releases/download/v0.2.1/orc-linux_amd64.duckdb_extension orc-linux_amd64.duckdb_extens 100%[] 10.69M 7.56MB/s in 1.4s 2026-09-02 18:41:56 (7.56 MB/s) - orc-linux_amd64.duckdb_extension saved [11206414/11206414]DuckDB v2.0.0-alpha有些版本没有插件不建议使用。C:\dduckdb0831 DuckDB v2.0.0-alpha39651 (Cyanoptera) Enter .help for usage hints. memory D load tpch; IO Error: Extension C:\Users\lt\.duckdb\extensions\v2.0.0-alpha39651\windows_amd64\tpch.duckdb_extension not found. Extension tpch is an existing extension. Install it first using INSTALL tpch. memory D install tpch; HTTP Error: Failed to download extension tpch at URL http://extensions.duckdb.org/v2.0.0-alpha39651/windows_amd64/tpch.duckdb_extension.gz (HTTP 404) Extension tpch is an existing extension. For more info, visit https://duckdb.org/docs/current/extensions/troubleshooting?versionv2.0.0-alpha39651platformwindows_amd64extensiontpch memory D .exit建议使用正式版本如DuckDB v1.5.5。C:\dduckdb155 DuckDB v1.5.5 (Variegata) Enter .help for usage hints. memory D load tpch; IO Error: Extension C:\Users\lt\.duckdb\extensions\v1.5.5\windows_amd64\tpch.duckdb_extension not found. Extension tpch is an existing extension. Install it first using INSTALL tpch. memory D install tpch; memory D load tpch; memory D call dbgen(sf2); ┌─────────┐ │ Success │ │ boolean │ └─────────┘ 0 rows memory D .timer on memory D .tables ────────────────────────────────────────────── memory ─────────────────────────────────────────────── ─────────────────────────────────────────────── main ──────────────────────────────────────────────── ┌─────────────────────────┐┌───────────────────────┐┌─────────────────────────┐┌──────────────────────┐ │ lineitem ││ part ││ orders ││ customer │ │ ││ ││ ││ │ │ l_orderkey bigint ││ p_partkey bigint ││ o_orderkey bigint ││ c_custkey bigint │ │ l_partkey bigint ││ p_name varchar ││ o_custkey bigint ││ c_name varchar │ │ l_suppkey bigint ││ p_mfgr varchar ││ o_orderstatus varchar ││ c_address varchar │ │ l_linenumber bigint ││ p_brand varchar ││ o_totalprice decimal ││ c_nationkey integer │ │ l_quantity decimal ││ p_type varchar ││ o_orderdate date ││ c_phone varchar │ │ l_extendedprice decimal ││ p_size integer ││ o_orderpriority varchar ││ c_acctbal decimal │ │ l_discount decimal ││ p_container varchar ││ o_clerk varchar ││ c_mktsegment varchar │ │ l_tax decimal ││ p_retailprice decimal ││ o_shippriority integer ││ c_comment varchar │ │ l_returnflag varchar ││ p_comment varchar ││ o_comment varchar ││ │ │ l_linestatus varchar ││ ││ ││ 300000 rows │ │ l_shipdate date ││ 400000 rows ││ 3.00 million rows │└──────────────────────┘ │ l_commitdate date │└───────────────────────┘└─────────────────────────┘ │ l_receiptdate date │ │ l_shipinstruct varchar │ │ l_shipmode varchar │ │ l_comment varchar │ │ │ │ 12.00 million rows │ └─────────────────────────┘ ┌─────────────────────┐┌───────────────────────┐┌─────────────────────┐┌─────────────────────┐ │ supplier ││ partsupp ││ nation ││ region │ │ ││ ││ ││ │ │ s_suppkey bigint ││ ps_partkey bigint ││ n_nationkey integer ││ r_regionkey integer │ │ s_name varchar ││ ps_suppkey bigint ││ n_name varchar ││ r_name varchar │ │ s_address varchar ││ ps_availqty bigint ││ n_regionkey integer ││ r_comment varchar │ │ s_nationkey integer ││ ps_supplycost decimal ││ n_comment varchar ││ │ │ s_phone varchar ││ ps_comment varchar ││ ││ 5 rows │ │ s_acctbal decimal ││ ││ 25 rows │└─────────────────────┘ │ s_comment varchar ││ 1.60 million rows │└─────────────────────┘ │ │└───────────────────────┘ │ 20000 rows │ └─────────────────────┘ memory D select sum(l_linenumber) from lineitem; ┌───────────────────┐ │ sum(l_linenumber) │ │ int128 │ ├───────────────────┤ │ 35994729 │ │ (35.99 million) │ └───────────────────┘ Run Time (s): real 0.011 user 0.015625 sys 0.000000 memory D copy (select l_linenumber from lineitem) to lll.parquet; Run Time (s): real 0.061 user 0.515625 sys 0.078125 memory D copy (select COLUMNS(* EXCLUDE l_comment) from lineitem) to lll.parquet; Run Time (s): real 1.788 user 7.234375 sys 10.890625先安装tpch插件生成8个表规模为2G先测试内置写入parquet文件。安装crc插件要注意如果已经装过旧版要force install。memory D install ./orc.duckdb_extension; Invalid Input Error: Installing extension orc failed. The extension is already installed but the origin is different. Currently installed extension is from repository http://community-extensions.duckdb.org, while the extension to be installed is from custom_path ./orc.duckdb_extension. To solve this rerun this command with FORCE INSTALL memory D force install ./orc.duckdb_extension; memory D load orc; memory D call dbgen(sf2); ┌─────────┐ │ Success │ │ boolean │ └─────────┘ 0 rows memory D .timer on memory D copy (select COLUMNS(* EXCLUDE l_comment) from lineitem) to lll.arc; Run Time (s): real 2.115 user 11.062500 sys 1.328125 memory D copy (select COLUMNS(* EXCLUDE l_comment) from lineitem) to lll.orc; Binder Error: write_orc: type Decimal128(38, 2) not supported by the ORC writer yet (supported: bool, int8-64, float32/64, date, timestamp, varchar, blob) Run Time (s): real 0.006 user 0.000000 sys 0.000000 memory D注意写错后缀arc默认写入csv文件。写对orc后缀后插件检测出不兼容的小数类型Decimal128报错退出。利用下面的脚本转换为支持的double类型再写入orc文件。 sql copy( select l_orderkey , l_partkey , l_suppkey , l_linenumber , l_quantity ::double l_quantity , l_extendedprice ::double l_extendedprice , l_discount ::double l_discount , l_tax ::double l_tax , l_returnflag , l_linestatus , l_shipdate , l_commitdate , l_receiptdate , l_shipinstruct , l_shipmode , from lineitem) to lll.orc;写入orc文件用时约是parquet格式的3倍。memory D copy( select l_orderkey , l_partkey , l_suppkey , l_linenumber , l_quantity ::double l_quantity , l_extendedprice ::double l_extendedprice , l_discount ::double l_discount , l_tax ::double l_tax , l_returnflag , l_linestatus , l_shipdate , l_commitdate , l_receiptdate , l_shipinstruct , l_shipmode , from lineitem) to lll.orc; Run Time (s): real 4.808 user 4.468750 sys 0.406250 memory D copy( select l_orderkey , l_partkey , l_suppkey , l_linenumber , l_quantity ::double l_quantity , l_extendedprice ::double l_extendedprice , l_discount ::double l_discount , l_tax ::double l_tax , l_returnflag , l_linestatus , l_shipdate , l_commitdate , l_receiptdate , l_shipinstruct , l_shipmode , from lineitem) to lll.parquet; Run Time (s): real 1.619 user 6.734375 sys 10.250000查询文件的时间差距更大相差30倍。注意orc插件写的文件列名被修改为了column_的形式这是DuckDB C API的限制。memory D select sum(l_linenumber) from lll.parquet; ┌───────────────────┐ │ sum(l_linenumber) │ │ int128 │ ├───────────────────┤ │ 35994729 │ │ (35.99 million) │ └───────────────────┘ Run Time (s): real 0.032 user 0.000000 sys 0.000000 memory D select sum(l_linenumber) from lll.orc; Binder Error: Referenced column l_linenumber not found in FROM clause! Candidate bindings: column_0, column_1, column_2, column_3, column_4 LINE 1: select sum(l_linenumber) from lll.orc; ^ Run Time (s): real 0.012 user 0.000000 sys 0.000000 memory D select sum(column_4) from lll.orc; ┌──────────────────┐ │ sum(column_4) │ │ double │ ├──────────────────┤ │ 305976330.0 │ │ (305.98 million) │ └──────────────────┘ Run Time (s): real 1.217 user 2.031250 sys 2.359375 memory D select sum(column_3) from lll.orc; ┌─────────────────┐ │ sum(column_3) │ │ int128 │ ├─────────────────┤ │ 35994729 │ │ (35.99 million) │ └─────────────────┘ Run Time (s): real 1.137 user 2.046875 sys 1.515625 memory D是否这种文件格式天生就慢不一定比如clickhouse两者的查询时间只差1倍可能是因为orc插件不支持并行。C:\dwsl rootDESKTOP-59T6U68:/mnt/c/d# ./clickhouse ClickHouse local version 26.8.1.1157 (official build). :) select sum(column_3) from file(lll.orc); SELECT sum(column_3) FROM file(lll.orc) Query id: 4426b39b-b538-48f3-a265-37e53449ae3d ┌─sum(column_3)─┐ 1. │ 35994729 │ -- 35.99 million └───────────────┘ 1 row in set. Elapsed: 0.112 sec. Processed 12.00 million rows, 254.23 MB (106.81 million rows/s., 2.26 GB/s.) Peak memory usage: 9.86 MiB. :) select sum(l_linenumber) from file(lll.parquet); SELECT sum(l_linenumber) FROM file(lll.parquet) Query id: fe9eef68-0d56-46d9-88e4-d1027a6bc423 ┌─sum(l_linenumber)─┐ 1. │ 35994729 │ -- 35.99 million └───────────────────┘ 1 row in set. Elapsed: 0.064 sec. Processed 12.00 million rows, 2.94 MB (187.68 million rows/s., 46.02 MB/s.) Peak memory usage: 47.16 MiB.