CSV(Comma-Separated Values)

永井 忠一 2024.10.17


Perl DBI

DBD::CSV モジュールを利用。インストール

Linux apt
$ sudo apt install libdbi-perl libdbd-csv-perl

例。SQL により、CSV ファイルを扱う

CREATE TABLE
use strict;
use warnings;

use DBI;

my $dbh = DBI->connect("DBI:CSV:f_dir=./") or die;

my $sth = $dbh->prepare("CREATE TABLE test.csv (id INTEGER, data REAL, name text)") or die;
$sth->execute() or die;

$sth->finish();
$dbh->disconnect();
INSERT
use strict;
use warnings;

use DBI;

my $dbh = DBI->connect("DBI:CSV:f_dir=./") or die;

my $sth = $dbh->prepare("INSERT INTO test.csv (id, data, name) VALUES(?, ?, ?)") or die;
$sth->execute(1, 1.1, "test write 1") or die;
$sth->execute(2, 2.2, "test write 2") or die;
$sth->execute(3, 3.3, "test write 3") or die;

$sth->finish();
$dbh->disconnect();
SELECT
use strict;
use warnings;

use DBI;

my $dbh = DBI->connect("DBI:CSV:f_dir=./") or die;

my $sth = $dbh->prepare("SELECT id, data, name FROM test.csv") or die;
$sth->execute() or die;
while (my $row = $sth->fetchrow_hashref()) {
    print($row->{"id"} . "|" .  $row->{"data"} . "|" . $row->{"name"} . "\n");
}

$sth->finish();
$dbh->disconnect();
UPDATE
use strict;
use warnings;

use DBI;

my $dbh = DBI->connect("DBI:CSV:f_dir=./") or die;

$dbh->do("UPDATE test.csv SET data = data + 0.1 -- adhoc") or die;

$dbh->disconnect();
DELETE
use strict;
use warnings;

use DBI;

my $dbh = DBI->connect("DBI:CSV:f_dir=./") or die;

my $sth = $dbh->prepare("DELETE FROM test.csv WHERE id = ?") or die;
$sth->execute(3) or die;

$sth->finish();
$dbh->disconnect();
DROP TABLE
use strict;
use warnings;

use DBI;

my $dbh = DBI->connect("DBI:CSV:f_dir=./") or die;

my $sth = $dbh->prepare("DROP TABLE IF EXISTS test.csv") or die;
$sth->execute() or die;

$sth->finish();
$dbh->disconnect();

(データソースにはディレクトリを指定する。テーブルは、ファイルとなる)

ドキュメント


DataFrame によるアクセス

CSV ファイルへの書き出し、CSV ファイルから DataFrame の構築

IPython
$ ipython3
Python 3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.20.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pandas as pd
   ...: pd.DataFrame({'id': [1, 2, 3], 'data': [1.1, 2.2, 3.3], 'name': ['record 1', 'record 2', 'record 3']}).to_csv('test.csv', index=False)

In [2]: pd.read_csv('test.csv')
Out[2]:
   id  data      name
0   1   1.1  record 1
1   2   2.2  record 2
2   3   3.3  record 3

ドキュメント


ファイルアクセス

(レコード単位での処理)

Perl と Python

Perl モジュールのインストール(Python では、csv は標準ライブラリ)

Linux apt
$ sudo apt install libtext-csv-xs-perl libtext-csv-perl

PerlPython
use strict;
use warnings;

use Text::CSV_XS;

my $csv = Text::CSV_XS->new({ binary => 1 }) or die;

open(my $fh, ">", "test.csv") or die;

if (1) { # header
    $csv->combine(("id", "data", "name"));
    print($fh $csv->string() . "\n");
}

$csv->combine((1, 1.1, "record 1")); print($fh $csv->string() . "\n");
$csv->combine((2, 2.2, "record 2")); print($fh $csv->string() . "\n");
$csv->combine((3, 3.3, "record 3")); print($fh $csv->string() . "\n");

close($fh);
import csv

with open('test.csv', 'w') as f:
    writer = csv.writer(f)
    if True: # header
        writer.writerow(['id', 'data', 'name'])
    writer.writerow([1, 1.1, 'record 1'])
    writer.writerow([2, 2.2, 'record 2'])
    writer.writerow([3, 3.3, 'record 3'])
use strict;
use warnings;

use Text::CSV_XS;

my $csv = Text::CSV_XS->new({ binary => 1 }) or die;

open(my $fh, "<", "test.csv") or die;

if (1) {
    $csv->parse(scalar <$fh>) or die;
    my @header = $csv->fields();
    print("@header\n");
}

while (<$fh>) {
    $csv->parse($_) or die;
    my @row = $csv->fields();
    print("@row\n");
}

close($fh);
readerDictReader
import csv

with open('test.csv') as f:
    reader = csv.reader(f)
    if True:
        header = next(reader)
        assert type(header) == list
        print(header)
    for row in reader:
        assert type(row) == list
        print(row)
import csv

with open('test.csv') as f:
    reader = csv.DictReader(f)
    for row in reader:
        assert type(row) == dict
        print(row['id'], row['data'], row['name'])

ドキュメント


データベースの CSV サポート

《CSV ファイルへのエクスポート、CSV ファイルからのインポートについて》


© 2024 Tadakazu Nagai