This post is older than a year. Consider some information might not be accurate anymore.
logstash allows with various plugins a quick solution, to convert CSV input to JSON output.
This test configuration demonstrates a simple example with nested properties.
# convert from csv to json
input {
generator {
type => 'csv'
message => '2015-10-08,CH,4711'
count => 1
}
}
filter {
csv {
columns => ['date', 'countryCode', 'value']
}
mutate {
rename => {
"date" => "[payment][date]"
"countryCode" => "[payment][origin]"
"value" => "[payment][value]"
}
}
}
output {
stdout {
codec => rubydebug{}
}
}
output {
stdout {
codec => json{}
}
}
- With the generator input plugin, we generate a CSV line.
- The csv filter plugin, allows the parsing of the CSV data.
- The mutate filter plugin, allows with rename to nest the properties under payment.
- The output contains two definitions, the rubydebug and the JSON output (pretty printed by me).
The output
Logstash startup completed
{
"message" => [
[0] "2015-10-08,CH,4711"
],
"@version" => "1",
"@timestamp" => "2015-10-08T14:31:03.344Z",
"type" => "csv",
"host" => "cinhtau",
"sequence" => 0,
"payment" => {
"date" => "2015-10-08",
"origin" => "CH",
"value" => "4711"
}
}
{
"message":["2015-10-08,CH,4711"],
"@version":"1",
"@timestamp":"2015-10-08T14:31:03.344Z",
"type":"csv",
"host":"cinhtau",
"sequence":0,
"payment":{
"date":"2015-10-08",
"origin":"CH",
"value":"4711"
}
}
Logstash shutdown completed