graylog-plugin-redis-lookup/README.md

# RedisLookupPlugin Plugin for Graylog

Plugin to add Redis Data Adapter in read/write to graylog so you can store and retrieve key/values and lists of string values from pipelines  
Support Redis authentication (with password and username/password)  

**Required Graylog version:** 5.0 and later

Installation
------------

[Download the plugin](https://git.nosd.in/yo/graylog-redis-lookup-plugin/releases/download/v1.0.3/graylog-plugin-redis-lookup-1.0.3.jar)
and place the `.jar` file in your Graylog plugin directory. The plugin directory
is the `plugins/` folder relative from your `graylog-server` directory by default
and can be configured in your `graylog.conf` file.

Restart `graylog-server` and you are done.

Development
-----------

You can improve your development experience for the web interface part of your plugin
dramatically by making use of hot reloading. To do this, do the following:

* `git clone https://github.com/Graylog2/graylog2-server.git`
* `cd graylog2-server/graylog2-web-interface`
* `ln -s $YOURPLUGIN plugin/`
* `npm install && npm start`

Usage
-----
### About data types
* "Strings" lookup table will use GET/SET redis commands. Designed to set, get, expire, delete keys.  
* "Streams" is designed for a "fire-and-forget" usage. Only "lookup_set_value" will work. Please note there is currently no TTL on stream, so you need to have an external process to purge your stream if you do not want it to grow indefinitely.

### Usage in pipelines

* Create data adapter, cache (or not), lookup table  
* Use 'lookup_set_value(lookup_table, key, value, [ttl])' to create or update key in redis  
* Use 'lookup_value(lookup_table, key)' to get key value  
* Use 'lookup_clear_key(lookup_table, key)' to remove key  
* Use 'lookup_has_value(lookup_table, key)' to test key existence  
* Use 'lookup_assign_ttl(lookup_table, key, ttl)' to change TTL of existing key  
* Use 'lookup_set_string_list(lookup_table, key, value, [ttl])' to create a list named "key"  
* Use 'lookup_add_string_list(lookup_table, key, value, [keep_duplicates])' to add value list to existing list  
* Use 'lookup_remove_string_list(lookup_table, key, value) to remove a string from list "key"  

By default single value keys will be created in Redis with the default TTL defined at data adapter creation time  

Be aware that only setting TTL with 'lookup_assign_ttl' of 'lookup_set_string_list' alter TTL value in Redis ; so a list created with a TTL of 3600 will expire in 3600 seconds, even if it was updated with 'lookup_add_string_list' some seconds before expiration.  

Known bugs
----------

* Deletion via lookup_remove_string_list, lookup_clear_key or keep_duplicates=false sometimes not done.  
* lookup_add_string_list : Each character of a string list is considered an item  

Getting started
---------------

This project is using Maven 3 and requires Java 8 or higher.

* Clone this repository.
* Run `mvn package` to build a JAR file.
* Optional: Run `mvn jdeb:jdeb` and `mvn rpm:rpm` to create a DEB and RPM package respectively.
* Copy generated JAR file in target directory to your Graylog plugin directory.
* Restart the Graylog.

Plugin Release
--------------

We are using the maven release plugin:

```
$ mvn release:prepare
[...]
$ mvn release:perform
```

This sets the version numbers, creates a tag and pushes to GitHub. Travis CI will build the release artifacts and upload to GitHub automatically.