User Tools

Site Tools


newdevs:code:import_csv_into_db

This is an old revision of the document!


Import Data from a CSV file into database tables

The Concerto database isn't very large, and sometimes you need to test things with more data. If you can pull and anonymize production-level data (but not too much to crash your test server), you can copy it into place with a CSV file.

1. Log into the database as the root user.

2. Create a placeholder schema for your project.

CREATE SCHEMA tlittle;

3. Create a placeholder table for your data to start in, and name the columns–including their datatype–for your table.

create table tlittle.acqprolist 
(name text,
owner integer,
currency_type text,
code text, 
active boolean,
prepayment_required boolean,
default_copy_count integer);

Make sure to include, at minimum, any required fields in your final table. This example is based on acq.provider, so these are the required fields in acq.provider. Each name of the column is followed by its datatype, e.g. prepayment_required (column) boolean (datatype).

4. Now we're going to populate our starter table from our CSV file. Make sure your CSV file is on the server somewhere you know.

copy tlittle.acqprolist from '/home/tlittle/pines_acqprolist.csv' csv header;

Then if you do a select * from tlittle.acqprolist you'll see it's now populated with the data from our CSV file.

5. To get that data into our Evergreen table, we need to copy it over.

insert into acq.provider (name,
owner,
currency_type,
code,
active,
prepayment_required,
default_copy_count) select 
name,
owner,
currency_type,
code,
active,
prepayment_required,
default_copy_count from tlittle.acqprolist;

So the second half is the SELECT–select all these from tlittle.acqprolist–which we then INSERT with the first portion.

newdevs/code/import_csv_into_db.1677521686.txt.gz · Last modified: 2023/02/27 13:14 by tlittle

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki

© 2008-2022 GPLS and others. Evergreen is open source software, freely licensed under GNU GPLv2 or later.
The Evergreen Project is a U.S. 501(c)3 non-profit organization.