fully support the json format in http output
Description
Gliffy Diagrams
Activity

Jose M. G. March 13, 2013 at 6:27 PM
ok, thank you very much for your help and comments.
Jordan Sissel March 13, 2013 at 6:12 PM
Once more outputs use Stud::Buffer, emitting an array of events to any output will be easier. Possible http can be one of the early candidates for using this.

Jose M. G. March 13, 2013 at 1:59 PM
Hi,
I think you know better than anyone what the best option for these suggestions.
That said, what I would do is simply by default add the brackets to the outside json format like to send a list of objects but only one.
And somehow if someone has the need to send another object would only be necessary to add it to the first putting it below, or by some identifying mark start of object.
Do not know if this makes sense for you.
Regards.

John E. Vincent March 13, 2013 at 1:36 PM
Thanks for opening this ticket.
The biggest issue that this presents is that logstash largely operates on individual events at a time. Obviously that has its plusses (no need to maintain state between events) but also its drawbacks (no batching).
Having said that, adding batching to the http output COULD be possible but doesn't make sense for anything OTHER than the JSON format (how would we batch http messages outside of the JSON format for instance).
I think initially this would likely be a NEW output plugin until the semantics of how to handle the configuration of a universal http plugin gets noodled out. To clarify, the problem becomes something like this:
Look at the current options for the http output (http://logstash.net/docs/1.1.9/outputs/http)
Under the "format" option, how would you suggest batching the "post" style of formatting?
If we had an ADDITIONAL configuration (something like batch_size), it could get confusing if it ONLY applied to the JSON format
We could call the the new config options something like json_batch_size but that gets a bit cumbersome as you add more tweaks to the json-only format in the output plugin.
I like the idea (and I think we use it in the librato output) but I'm curious if there's a way we can address the 'post' format as well.
Details
Details
Assignee

Reporter

Would be nice if the output http module fully supports json format as specified in RFC 4627 http://www.ietf.org/rfc/rfc4627.txt :
Currently it is only possible to send an event object like this:
{ "payloadData" : ["a...@org1.com", "120d", "abc", "Complex Event Processor", "fail", "Data Team"] , "metaData" : ["192.168.1.1"] }
and the RFC specifies the ability to send a list of objects (one or more but always with square brackets)
[ { "payloadData" : ["a...@org1.com", "120d", "abc", "Complex Event Processor", "fail", "Data Team"] , "metaData" : ["192.168.1.1"] } ]
RFC4627 example :
This is a JSON array containing two objects:
[
{
"precision": "zip",
"Latitude": 37.7668,
"Longitude": -122.3959,
"Address": "",
"City": "SAN FRANCISCO",
"State": "CA",
"Zip": "94107",
"Country": "US"
},
{
"precision": "zip",
"Latitude": 37.371991,
"Longitude": -122.026020,
"Address": "",
"City": "SUNNYVALE",
"State": "CA",
"Zip": "94085",
"Country": "US"
}
]
I think json is a very important format for data exchange and it will become more and more important every day.