Encryption support options

Description

I need to get the ball rolling on encryption options for Logstash as it will soon be a need internally for my company.

There are two types of encryption to consider:

  • transport level

  • message level

Transport level would require customization on input and output plugins of appropriate types. This might fit with an option in base.rb for each or it might not. Notable conflicts include rabbitmq.

Message level security COULD be implemented at the ingress/egress but probably makes the most sense as a filter. In such a case I envision that this encryption require specifying fields that the user wishes to encrypt. This would require the user to make decryption the first step in any filter chains to do further processing.

Another option is to provide for a keyed field name in the json_event format that indicated decryption needed to happen automatically. Possibly this would be something like this:

{ "crypted_fields":["message", "source_host", "foo"] }

Upon seeing this, logstash would automatically decrypt the relevant fields before passing up the chain. Obviously, there needs to be a way for logstash to know how to decrypt it. In this basic implementation, a shared key would probably suffice:

input { stdin { type => "stdin-test" shared_key => "makemetastedagoat" } }

Obviously the keys would need to match up and this is only a basic impl.

Comments?

Activity

Show:
John E. Vincent
February 22, 2012, 5:42 AM

There's a basic filter implementation here: http://goo.gl/3Ru2p

It requires that the encrypt be the last filter before output and decrypt be the first filter before input.

Jan Seidl
March 1, 2012, 3:40 PM

John I think this approach is OK but it can be replaced by AMQP servers running in SSL/TLS ?

John E. Vincent
March 1, 2012, 5:57 PM

That only resolves the case of using AMQP. All logstash plugins should support (where appropriate) both encryption in transit and, if possible, at rest. The filters can address the 'at rest' part. The inputs/outputs will handle the 'in transit' part.

Jan Seidl
March 1, 2012, 8:06 PM

I see your point. So you could get encrypted messages through tcp filters and such. I think its reasonable to have it as fitler and your implementation is fine, we should just address use of different encryption types and lenghts and even PKI.

I can help you on this if you want =)

Assignee

John E. Vincent

Reporter

John E. Vincent

Labels

None

Fix versions

Affects versions

Configure