90,301 questions
Best practices
0
votes
1
replies
74
views
How to incrementally merge daily CSV delta updates into a master file without losing expired records?
I am building a data pipeline to create a cumulative master dataset from daily CSV exports generated by a booking system.
Context & Data Structure
Source: A daily CSV file containing status ...
- 1
Best practices
0
votes
0
replies
46
views
CSV Column Structure Evolution Snowflake
I'm stuck with a client's request.
Context: Ingestion of CSV files from a S3 bucket to a table in Snowflake. In addition, I need to add three columns, one with the current date, one with the name of ...
0
votes
0
answers
63
views
Azure Fabric - Lakehouse SQL endpoint - CSV with non UTF codepage
I'm trying to use OPENROWSET over a very simple CSV file with only 3 columns
col1;col2;col3
FRANK;998533805;20.06
FINSÅS KARL;999999;10.05
RØNNING FRANK KARL;999999;7.05
but since the encoding of the ...
- 23
3
votes
3
answers
159
views
Write multi-columns to CSV with for loop
This code snippet contains bpy. I am trying to write data to a three column CSV. The first column is the vertex index, the second column is the U coordinate, and the third column is the V coordinate.
...
0
votes
0
answers
59
views
CsvValidator optional header column
I am using the UK National Archives CSV Validator via the Maven dependency uk.gov.nationalarchives:csv-validator-java-api-1.4.0.jar to validate both the headers and the body of CSV/txt files.
I am ...
- 29
1
vote
2
answers
115
views
download csv file from OSF
I am trying to download a .csv file from a Public Open Science Framework (OSF) project through my script on GitHub. I do not want to download a local copy.
I've tried using two different options for ...
- 1,539
3
votes
5
answers
211
views
Fusing row with previous row(s) depending on criteria
I'm trying to fix a CSV file I received in which actual new lines were inserted throughout the data, effectively splitting one row of data into multiple ones with incomplete columns. The line breaks ...
- 33
4
votes
3
answers
173
views
Python polars script leaks memory and crashes (scans CSV to lazyframes - writes to database)
I wrote a small script to scan a CSV in Python polars, select specific columns/filter specific rows in lazyframes and upload the result to a Postgres DB.
The script works with a smaller test CSV but, ...
Advice
0
votes
3
replies
90
views
How can I save a CSV file locally in a Flutter app so it persists within the app?
I'm building a Flutter app and I need to generate and save a CSV file from within the app. My goal is for the file to be stored locally on the device so that it persists between app launches and ...
Best practices
0
votes
3
replies
144
views
How to import and manipulate a CSV?
I have a CSV file. I need to perform two tasks:
How do I properly load this file into R so it becomes a data frame object?
Once loaded, how can I filter the rows or select specific columns?
What I ...
3
votes
3
answers
229
views
Import large CSV files with many unknown column names and types (auto-detection)
Using PostgreSQL 18, I want to import a large .csv file with unknown column names and types. I have 2 related .csv files:
The 1st file contains around 100 columns and 2,000,000 rows of data, like ...
- 61
-1
votes
1
answer
81
views
How can I specify a variable amount of whitespace as CSV import column separator?
In SQLite:
.separator " " # is one space
.separator , # is a comma
.separator "\t" # is a tab
But how can I specify a variable number of spaces for CSV import? E.g.:
<<...
- 600
5
votes
1
answer
133
views
MariaDB import csv file that doesn't have commas at the end of lines [closed]
I'm trying to import a CSV file to a MariaDB table using this query:
load data local infile 'C:\\Users\\Kevin\\Downloads\\TCGToday.csv'
into table tcgtoday
fields terminated by ','
optionally enclosed ...
- 53
0
votes
1
answer
145
views
Trouble getting R to read data values as numeric [closed]
I am trying to run analysis on salmon aquaculture mortality data to identify events where mortality of >7% of stock was lost, however when I try to get R to read the values in my mortality column I ...
- 11
-2
votes
0
answers
40
views
Change Delimiter Type in SPSS so that it can be converted to CSV?
does anybody know how to change delimiter in SPSS settings so that when i convert it to csv, the new delimiter applied automatically? or is there any other easier way?
