first commit

This commit is contained in:
yo000 2024-02-03 14:07:41 +01:00
commit 1ff3f557a7
39 changed files with 165488 additions and 0 deletions

4
.gitignore vendored Normal file
View File

@ -0,0 +1,4 @@
node_modules/*
target/*

10
.mvn/jvm.config Normal file
View File

@ -0,0 +1,10 @@
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
--add-opens jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED
--add-opens jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED

27
.travis.yml Normal file
View File

@ -0,0 +1,27 @@
sudo: required
dist: trusty
language: java
jdk:
- oraclejdk8
addons:
apt:
packages:
- rpm
before_deploy:
- mvn jdeb:jdeb && export RELEASE_DEB_FILE=$(ls target/*.deb)
- mvn rpm:rpm && export RELEASE_RPM_FILE=$(find target/ -name '*.rpm' | tail -1)
- rm -f target/original-*.jar
- export RELEASE_PKG_FILE=$(ls target/*.jar)
- echo "Deploying release to GitHub releases"
deploy:
provider: releases
api_key:
secure: <enter your encrypted GitHub access token>
file:
- "${RELEASE_PKG_FILE}"
- "${RELEASE_DEB_FILE}"
- "${RELEASE_RPM_FILE}"
skip_cleanup: true
on:
tags: true
jdk: oraclejdk8

26
GETTING-STARTED.md Normal file
View File

@ -0,0 +1,26 @@
Getting started with your new Graylog plugin
============================================
Welcome to your new Graylog plugin!
Please refer to https://docs.graylog.org/docs/plugins for documentation on how to write
plugins for Graylog.
Travis CI
---------
There is a `.travis.yml` template in this project which is prepared to automatically
deploy the plugin artifacts (JAR, DEB, RPM) to GitHub releases.
You just have to add your encrypted GitHub access token to the `.travis.yml`.
The token can be generated in your [GitHub personal access token settings](https://github.com/settings/tokens).
Before Travis CI works, you have to enable it. Install the Travis CI command line
application and execute `travis enable`.
To encrypt your GitHub access token you can use `travis encrypt`.
Alternatively you can use `travis setup -f releases` to automatically create a GitHub
access token and add it to the `.travis.yml` file. **Attention:** doing this
will replace some parts of the `.travis.yml` file and you have to restore previous
settings.

58
README.md Normal file
View File

@ -0,0 +1,58 @@
# RedisLookupPlugin Plugin for Graylog
[![Build Status](https://travis-ci.org/https://git.nosd.in/yo/graylog-plugin-redis-lookup.svg?branch=master)](https://travis-ci.org/https://git.nosd.in/yo/graylog-plugin-redis-lookup)
__Use this paragraph to enter a description of your plugin.__
**Required Graylog version:** 2.0 and later
Installation
------------
[Download the plugin](https://github.com/https://git.nosd.in/yo/graylog-plugin-redis-lookup/releases)
and place the `.jar` file in your Graylog plugin directory. The plugin directory
is the `plugins/` folder relative from your `graylog-server` directory by default
and can be configured in your `graylog.conf` file.
Restart `graylog-server` and you are done.
Development
-----------
You can improve your development experience for the web interface part of your plugin
dramatically by making use of hot reloading. To do this, do the following:
* `git clone https://github.com/Graylog2/graylog2-server.git`
* `cd graylog2-server/graylog2-web-interface`
* `ln -s $YOURPLUGIN plugin/`
* `npm install && npm start`
Usage
-----
__Use this paragraph to document the usage of your plugin__
Getting started
---------------
This project is using Maven 3 and requires Java 8 or higher.
* Clone this repository.
* Run `mvn package` to build a JAR file.
* Optional: Run `mvn jdeb:jdeb` and `mvn rpm:rpm` to create a DEB and RPM package respectively.
* Copy generated JAR file in target directory to your Graylog plugin directory.
* Restart the Graylog.
Plugin Release
--------------
We are using the maven release plugin:
```
$ mvn release:prepare
[...]
$ mvn release:perform
```
This sets the version numbers, creates a tag and pushes to GitHub. Travis CI will build the release artifacts and upload to GitHub automatically.

22
build.config.js Normal file
View File

@ -0,0 +1,22 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
const path = require('path');
module.exports = {
// Make sure that this is the correct path to the web interface part of the Graylog server repository.
web_src_path: path.resolve(__dirname, '../graylog2-server', 'graylog2-web-interface'),
};

1128
dependency-reduced-pom.xml Normal file

File diff suppressed because it is too large Load Diff

BIN
node/node Executable file

Binary file not shown.

26
node/yarn/dist/LICENSE vendored Normal file
View File

@ -0,0 +1,26 @@
BSD 2-Clause License
For Yarn software
Copyright (c) 2016-present, Yarn Contributors. All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

60
node/yarn/dist/README.md vendored Normal file
View File

@ -0,0 +1,60 @@
<p align="center">
<a href="https://yarnpkg.com/">
<img alt="Yarn" src="https://github.com/yarnpkg/assets/blob/master/yarn-kitten-full.png?raw=true" width="546">
</a>
</p>
<p align="center">
Fast, reliable, and secure dependency management.
</p>
<p align="center">
<a href="https://circleci.com/gh/yarnpkg/yarn"><img alt="Circle Status" src="https://circleci.com/gh/yarnpkg/yarn.svg?style=shield&circle-token=5f0a78473b0f440afb218bf2b82323cc6b3cb43f"></a>
<a href="https://ci.appveyor.com/project/kittens/yarn/branch/master"><img alt="Appveyor Status" src="https://ci.appveyor.com/api/projects/status/0xdv8chwe2kmk463?svg=true"></a>
<a href="https://dev.azure.com/yarnpkg/yarn/_build"><img alt="Azure Pipelines status" src="https://dev.azure.com/yarnpkg/yarn/_apis/build/status/Yarn%20Acceptance%20Tests"></a>
<a href="https://discord.gg/yarnpkg"><img alt="Discord Chat" src="https://img.shields.io/discord/226791405589233664.svg"></a>
<a href="http://commitizen.github.io/cz-cli/"><img alt="Commitizen friendly" src="https://img.shields.io/badge/commitizen-friendly-brightgreen.svg"></a>
</p>
---
**Fast:** Yarn caches every package it has downloaded, so it never needs to download the same package again. It also does almost everything concurrently to maximize resource utilization. This means even faster installs.
**Reliable:** Using a detailed but concise lockfile format and a deterministic algorithm for install operations, Yarn is able to guarantee that any installation that works on one system will work exactly the same on another system.
**Secure:** Yarn uses checksums to verify the integrity of every installed package before its code is executed.
## Features
* **Offline Mode.** If you've installed a package before, then you can install it again without an internet connection.
* **Deterministic.** The same dependencies will be installed in the same exact way on any machine, regardless of installation order.
* **Network Performance.** Yarn efficiently queues requests and avoids request waterfalls in order to maximize network utilization.
* **Network Resilience.** A single request that fails will not cause the entire installation to fail. Requests are automatically retried upon failure.
* **Flat Mode.** Yarn resolves mismatched versions of dependencies to a single version to avoid creating duplicates.
* **More emojis.** 🐈
## Installing Yarn
Read the [Installation Guide](https://yarnpkg.com/en/docs/install) on our website for detailed instructions on how to install Yarn.
## Using Yarn
Read the [Usage Guide](https://yarnpkg.com/en/docs/usage) on our website for detailed instructions on how to use Yarn.
## Contributing to Yarn
Contributions are always welcome, no matter how large or small. Substantial feature requests should be proposed as an [RFC](https://github.com/yarnpkg/rfcs). Before contributing, please read the [code of conduct](CODE_OF_CONDUCT.md).
See [Contributing](https://yarnpkg.com/org/contributing/).
## Prior art
Yarn wouldn't exist if it wasn't for excellent prior art. Yarn has been inspired by the following projects:
- [Bundler](https://github.com/bundler/bundler)
- [Cargo](https://github.com/rust-lang/cargo)
- [npm](https://github.com/npm/cli)
## Credits
Thanks to [Sam Holmes](https://github.com/samholmes) for donating the npm package name!

35
node/yarn/dist/bin/yarn vendored Executable file
View File

@ -0,0 +1,35 @@
#!/bin/sh
argv0=$(echo "$0" | sed -e 's,\\,/,g')
basedir=$(dirname "$(readlink "$0" || echo "$argv0")")
case "$(uname -s)" in
Darwin) basedir="$( cd "$( dirname "$argv0" )" && pwd )";;
Linux) basedir=$(dirname "$(readlink -f "$0" || echo "$argv0")");;
*CYGWIN*) basedir=`cygpath -w "$basedir"`;;
*MSYS*) basedir=`cygpath -w "$basedir"`;;
esac
command_exists() {
command -v "$1" >/dev/null 2>&1;
}
if command_exists node; then
if [ "$YARN_FORCE_WINPTY" = 1 ] || command_exists winpty && test -t 1; then
winpty node "$basedir/yarn.js" "$@"
else
exec node "$basedir/yarn.js" "$@"
fi
ret=$?
# Debian and Ubuntu use "nodejs" as the name of the binary, not "node", so we
# search for that too. See:
# https://lists.debian.org/debian-devel-announce/2012/07/msg00002.html
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=614907
elif command_exists nodejs; then
exec nodejs "$basedir/yarn.js" "$@"
ret=$?
else
>&2 echo 'Yarn requires Node.js 4.0 or higher to be installed.'
ret=1
fi
exit $ret

2
node/yarn/dist/bin/yarn.cmd vendored Executable file
View File

@ -0,0 +1,2 @@
@echo off
node "%~dp0\yarn.js" %*

31
node/yarn/dist/bin/yarn.js vendored Executable file
View File

@ -0,0 +1,31 @@
#!/usr/bin/env node
/* eslint-disable no-var */
/* eslint-disable flowtype/require-valid-file-annotation */
'use strict';
var ver = process.versions.node;
var majorVer = parseInt(ver.split('.')[0], 10);
if (majorVer < 4) {
console.error('Node version ' + ver + ' is not supported, please use Node.js 4.0 or higher.');
process.exit(1); // eslint-disable-line no-process-exit
} else {
try {
require(__dirname + '/../lib/v8-compile-cache.js');
} catch (err) {
// We don't have/need this on legacy builds and dev builds
}
// Just requiring this package will trigger a yarn run since the
// `require.main === module` check inside `cli/index.js` will always
// be truthy when built with webpack :(
// `lib/cli` may be `lib/cli/index.js` or `lib/cli.js` depending on the build.
var cli = require(__dirname + '/../lib/cli');
if (!cli.autoRun) {
cli.default().catch(function(error) {
console.error(error.stack || error.message || error);
process.exitCode = 1;
});
}
}

2
node/yarn/dist/bin/yarnpkg vendored Executable file
View File

@ -0,0 +1,2 @@
#!/usr/bin/env node
require('./yarn.js');

2
node/yarn/dist/bin/yarnpkg.cmd vendored Executable file
View File

@ -0,0 +1,2 @@
@echo off
"%~dp0\yarn.cmd" %*

153544
node/yarn/dist/lib/cli.js vendored Executable file

File diff suppressed because one or more lines are too long

351
node/yarn/dist/lib/v8-compile-cache.js vendored Normal file
View File

@ -0,0 +1,351 @@
'use strict';
const Module = require('module');
const crypto = require('crypto');
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const os = require('os');
const hasOwnProperty = Object.prototype.hasOwnProperty;
//------------------------------------------------------------------------------
// FileSystemBlobStore
//------------------------------------------------------------------------------
class FileSystemBlobStore {
constructor(directory, prefix) {
const name = prefix ? slashEscape(prefix + '.') : '';
this._blobFilename = path.join(directory, name + 'BLOB');
this._mapFilename = path.join(directory, name + 'MAP');
this._lockFilename = path.join(directory, name + 'LOCK');
this._directory = directory;
this._load();
}
has(key, invalidationKey) {
if (hasOwnProperty.call(this._memoryBlobs, key)) {
return this._invalidationKeys[key] === invalidationKey;
} else if (hasOwnProperty.call(this._storedMap, key)) {
return this._storedMap[key][0] === invalidationKey;
}
return false;
}
get(key, invalidationKey) {
if (hasOwnProperty.call(this._memoryBlobs, key)) {
if (this._invalidationKeys[key] === invalidationKey) {
return this._memoryBlobs[key];
}
} else if (hasOwnProperty.call(this._storedMap, key)) {
const mapping = this._storedMap[key];
if (mapping[0] === invalidationKey) {
return this._storedBlob.slice(mapping[1], mapping[2]);
}
}
}
set(key, invalidationKey, buffer) {
this._invalidationKeys[key] = invalidationKey;
this._memoryBlobs[key] = buffer;
this._dirty = true;
}
delete(key) {
if (hasOwnProperty.call(this._memoryBlobs, key)) {
this._dirty = true;
delete this._memoryBlobs[key];
}
if (hasOwnProperty.call(this._invalidationKeys, key)) {
this._dirty = true;
delete this._invalidationKeys[key];
}
if (hasOwnProperty.call(this._storedMap, key)) {
this._dirty = true;
delete this._storedMap[key];
}
}
isDirty() {
return this._dirty;
}
save() {
const dump = this._getDump();
const blobToStore = Buffer.concat(dump[0]);
const mapToStore = JSON.stringify(dump[1]);
try {
mkdirpSync(this._directory);
fs.writeFileSync(this._lockFilename, 'LOCK', {flag: 'wx'});
} catch (error) {
// Swallow the exception if we fail to acquire the lock.
return false;
}
try {
fs.writeFileSync(this._blobFilename, blobToStore);
fs.writeFileSync(this._mapFilename, mapToStore);
} catch (error) {
throw error;
} finally {
fs.unlinkSync(this._lockFilename);
}
return true;
}
_load() {
try {
this._storedBlob = fs.readFileSync(this._blobFilename);
this._storedMap = JSON.parse(fs.readFileSync(this._mapFilename));
} catch (e) {
this._storedBlob = Buffer.alloc(0);
this._storedMap = {};
}
this._dirty = false;
this._memoryBlobs = {};
this._invalidationKeys = {};
}
_getDump() {
const buffers = [];
const newMap = {};
let offset = 0;
function push(key, invalidationKey, buffer) {
buffers.push(buffer);
newMap[key] = [invalidationKey, offset, offset + buffer.length];
offset += buffer.length;
}
for (const key of Object.keys(this._memoryBlobs)) {
const buffer = this._memoryBlobs[key];
const invalidationKey = this._invalidationKeys[key];
push(key, invalidationKey, buffer);
}
for (const key of Object.keys(this._storedMap)) {
if (hasOwnProperty.call(newMap, key)) continue;
const mapping = this._storedMap[key];
const buffer = this._storedBlob.slice(mapping[1], mapping[2]);
push(key, mapping[0], buffer);
}
return [buffers, newMap];
}
}
//------------------------------------------------------------------------------
// NativeCompileCache
//------------------------------------------------------------------------------
class NativeCompileCache {
constructor() {
this._cacheStore = null;
this._previousModuleCompile = null;
}
setCacheStore(cacheStore) {
this._cacheStore = cacheStore;
}
install() {
const self = this;
this._previousModuleCompile = Module.prototype._compile;
Module.prototype._compile = function(content, filename) {
const mod = this;
function require(id) {
return mod.require(id);
}
require.resolve = function(request) {
return Module._resolveFilename(request, mod);
};
require.main = process.mainModule;
// Enable support to add extra extension types
require.extensions = Module._extensions;
require.cache = Module._cache;
const dirname = path.dirname(filename);
const compiledWrapper = self._moduleCompile(filename, content);
// We skip the debugger setup because by the time we run, node has already
// done that itself.
const args = [mod.exports, require, mod, filename, dirname, process, global];
return compiledWrapper.apply(mod.exports, args);
};
}
uninstall() {
Module.prototype._compile = this._previousModuleCompile;
}
_moduleCompile(filename, content) {
// https://github.com/nodejs/node/blob/v7.5.0/lib/module.js#L511
// Remove shebang
var contLen = content.length;
if (contLen >= 2) {
if (content.charCodeAt(0) === 35/*#*/ &&
content.charCodeAt(1) === 33/*!*/) {
if (contLen === 2) {
// Exact match
content = '';
} else {
// Find end of shebang line and slice it off
var i = 2;
for (; i < contLen; ++i) {
var code = content.charCodeAt(i);
if (code === 10/*\n*/ || code === 13/*\r*/) break;
}
if (i === contLen) {
content = '';
} else {
// Note that this actually includes the newline character(s) in the
// new output. This duplicates the behavior of the regular
// expression that was previously used to replace the shebang line
content = content.slice(i);
}
}
}
}
// create wrapper function
var wrapper = Module.wrap(content);
var invalidationKey = crypto
.createHash('sha1')
.update(content, 'utf8')
.digest('hex');
var buffer = this._cacheStore.get(filename, invalidationKey);
var script = new vm.Script(wrapper, {
filename: filename,
lineOffset: 0,
displayErrors: true,
cachedData: buffer,
produceCachedData: true,
});
if (script.cachedDataProduced) {
this._cacheStore.set(filename, invalidationKey, script.cachedData);
} else if (script.cachedDataRejected) {
this._cacheStore.delete(filename);
}
var compiledWrapper = script.runInThisContext({
filename: filename,
lineOffset: 0,
columnOffset: 0,
displayErrors: true,
});
return compiledWrapper;
}
}
//------------------------------------------------------------------------------
// utilities
//
// https://github.com/substack/node-mkdirp/blob/f2003bb/index.js#L55-L98
// https://github.com/zertosh/slash-escape/blob/e7ebb99/slash-escape.js
//------------------------------------------------------------------------------
function mkdirpSync(p_) {
_mkdirpSync(path.resolve(p_), parseInt('0777', 8) & ~process.umask());
}
function _mkdirpSync(p, mode) {
try {
fs.mkdirSync(p, mode);
} catch (err0) {
if (err0.code === 'ENOENT') {
_mkdirpSync(path.dirname(p));
_mkdirpSync(p);
} else {
try {
const stat = fs.statSync(p);
if (!stat.isDirectory()) { throw err0; }
} catch (err1) {
throw err0;
}
}
}
}
function slashEscape(str) {
const ESCAPE_LOOKUP = {
'\\': 'zB',
':': 'zC',
'/': 'zS',
'\x00': 'z0',
'z': 'zZ',
};
return str.replace(/[\\:\/\x00z]/g, match => (ESCAPE_LOOKUP[match]));
}
function supportsCachedData() {
const script = new vm.Script('""', {produceCachedData: true});
// chakracore, as of v1.7.1.0, returns `false`.
return script.cachedDataProduced === true;
}
function getCacheDir() {
// Avoid cache ownership issues on POSIX systems.
const dirname = typeof process.getuid === 'function'
? 'v8-compile-cache-' + process.getuid()
: 'v8-compile-cache';
const version = typeof process.versions.v8 === 'string'
? process.versions.v8
: typeof process.versions.chakracore === 'string'
? 'chakracore-' + process.versions.chakracore
: 'node-' + process.version;
const cacheDir = path.join(os.tmpdir(), dirname, version);
return cacheDir;
}
function getParentName() {
// `module.parent.filename` is undefined or null when:
// * node -e 'require("v8-compile-cache")'
// * node -r 'v8-compile-cache'
// * Or, requiring from the REPL.
const parentName = module.parent && typeof module.parent.filename === 'string'
? module.parent.filename
: process.cwd();
return parentName;
}
//------------------------------------------------------------------------------
// main
//------------------------------------------------------------------------------
if (!process.env.DISABLE_V8_COMPILE_CACHE && supportsCachedData()) {
const cacheDir = getCacheDir();
const prefix = getParentName();
const blobStore = new FileSystemBlobStore(cacheDir, prefix);
const nativeCompileCache = new NativeCompileCache();
nativeCompileCache.setCacheStore(blobStore);
nativeCompileCache.install();
process.once('exit', code => {
if (blobStore.isDirty()) {
blobStore.save();
}
nativeCompileCache.uninstall();
});
}
module.exports.__TEST__ = {
FileSystemBlobStore,
NativeCompileCache,
mkdirpSync,
slashEscape,
supportsCachedData,
getCacheDir,
getParentName,
};

27
node/yarn/dist/package.json vendored Normal file
View File

@ -0,0 +1,27 @@
{
"name": "yarn",
"installationMethod": "tar",
"version": "1.22.17",
"license": "BSD-2-Clause",
"preferGlobal": true,
"description": "📦🐈 Fast, reliable, and secure dependency management.",
"resolutions": {
"sshpk": "^1.14.2"
},
"engines": {
"node": ">=4.0.0"
},
"repository": "yarnpkg/yarn",
"bin": {
"yarn": "./bin/yarn.js",
"yarnpkg": "./bin/yarn.js"
},
"scripts": {
"preinstall": ":; (node ./preinstall.js > /dev/null 2>&1 || true)"
},
"config": {
"commitizen": {
"path": "./node_modules/cz-conventional-changelog"
}
}
}

60
node/yarn/dist/preinstall.js vendored Normal file
View File

@ -0,0 +1,60 @@
// This file is a bit weird, so let me explain with some context: we're working
// to implement a tool called "Corepack" in Node. This tool will allow us to
// provide a Yarn shim to everyone using Node, meaning that they won't need to
// run `npm install -g yarn`.
//
// Still, we don't want to break the experience of people that already use `npm
// install -g yarn`! And one annoying thing with npm is that they install their
// binaries directly inside the Node bin/ folder. And Because of this, they
// refuse to overwrite binaries when they detect they don't belong to npm. Which
// means that, since the "yarn" Corepack symlink belongs to Corepack and not npm,
// running `npm install -g yarn` would crash by refusing to override the binary :/
//
// And thus we have this preinstall script, which checks whether Yarn is being
// installed as a global binary, and remove the existing symlink if it detects
// it belongs to Corepack. Since preinstall scripts run, in npm, before the global
// symlink is created, we bypass this way the ownership check.
//
// More info:
// https://github.com/arcanis/pmm/issues/6
if (process.env.npm_config_global) {
var cp = require('child_process');
var fs = require('fs');
var path = require('path');
try {
var targetPath = cp.execFileSync(process.execPath, [process.env.npm_execpath, 'bin', '-g'], {
encoding: 'utf8',
stdio: ['ignore', undefined, 'ignore'],
}).replace(/\n/g, '');
var manifest = require('./package.json');
var binNames = typeof manifest.bin === 'string'
? [manifest.name.replace(/^@[^\/]+\//, '')]
: typeof manifest.bin === 'object' && manifest.bin !== null
? Object.keys(manifest.bin)
: [];
binNames.forEach(function (binName) {
var binPath = path.join(targetPath, binName);
var binTarget;
try {
binTarget = fs.readlinkSync(binPath);
} catch (err) {
return;
}
if (binTarget.startsWith('../lib/node_modules/corepack/')) {
try {
fs.unlinkSync(binPath);
} catch (err) {
return;
}
}
});
} catch (err) {
// ignore errors
}
}

28
package.json Normal file
View File

@ -0,0 +1,28 @@
{
"name": "RedisLookupPlugin",
"version": "1.0.0-SNAPSHOT",
"description": "",
"repository": {
"type": "git",
"url": "https://git.nosd.in/yo/graylog-plugin-redis-lookup"
},
"scripts": {
"build": "webpack",
"lint": "eslint src",
"lint:path": "eslint",
"test": "jest"
},
"keywords": [
"graylog"
],
"author": "johan <johan@nosd.in>",
"license": "MIT",
"eslintConfig": {
"extends": "graylog"
},
"dependencies": {
},
"devDependencies": {
"graylog-web-plugin": "file:../graylog2-server/graylog2-web-interface/packages/graylog-web-plugin"
}
}

328
pom.xml Normal file
View File

@ -0,0 +1,328 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (C) 2020 Graylog, Inc.
This program is free software: you can redistribute it and/or modify
it under the terms of the Server Side Public License, version 1,
as published by MongoDB, Inc.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
Server Side Public License for more details.
You should have received a copy of the Server Side Public License
along with this program. If not, see
<http://www.mongodb.com/licensing/server-side-public-license>.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.graylog.plugins</groupId>
<artifactId>graylog-plugin-web-parent</artifactId>
<version>5.0.0</version>
<relativePath>../graylog2-server/graylog-plugin-parent/graylog-plugin-web-parent</relativePath>
</parent>
<groupId>in.nosd.redis</groupId>
<artifactId>graylog-plugin-redis-lookup</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>${project.artifactId}</name>
<description>Graylog ${project.artifactId} plugin.</description>
<url>https://www.graylog.org</url>
<developers>
<developer>
<name>johan</name>
<email>johan@nosd.in</email>
</developer>
</developers>
<scm>
<connection>scm:git:git@github.com:https://git.nosd.in/yo/graylog-plugin-redis-lookup.git</connection>
<developerConnection>scm:git:git@github.com:https://git.nosd.in/yo/graylog-plugin-redis-lookup.git</developerConnection>
<url>https://github.com/https://git.nosd.in/yo/graylog-plugin-redis-lookup</url>
<tag>HEAD</tag>
</scm>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
<!-- Plugins will not be deployed by default - set to `false` if you actually want to deploy it -->
<maven.deploy.skip>true</maven.deploy.skip>
<graylog.version>${project.parent.version}</graylog.version>
<graylog.plugin-dir>/usr/share/graylog-server/plugin</graylog.plugin-dir>
</properties>
<!-- <distributionManagement>
<snapshotRepository>
<id>sonatype-nexus-snapshots</id>
<name>Sonatype Nexus Snapshots</name>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
<repository>
<id>sonatype-nexus-staging</id>
<name>Nexus Release Repository</name>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
</distributionManagement>
<repositories>
--> <!-- to make our snapshot releases work with Travis et al -->
<!-- <repository>
<id>sonatype-nexus-snapshots</id>
<name>Sonatype Nexus Snapshots</name>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
<repository>
<id>sonatype-nexus-releases</id>
<name>Sonatype Nexus Releases</name>
<url>https://oss.sonatype.org/content/repositories/releases</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
-->
<dependencies>
<dependency>
<groupId>org.graylog2</groupId>
<artifactId>graylog2-server</artifactId>
<version>${graylog.version}</version>
<scope>provided</scope>
</dependency>
<!-- Include source from test-jar to reuse test classes. -->
<dependency>
<groupId>org.graylog2</groupId>
<artifactId>graylog2-server</artifactId>
<version>${graylog.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.auto.value</groupId>
<artifactId>auto-value</artifactId>
<version>${auto-value.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.graylog.autovalue</groupId>
<artifactId>auto-value-javabean</artifactId>
<version>${auto-value-javabean.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.lettuce</groupId>
<artifactId>lettuce-core</artifactId>
<version>6.3.0.RELEASE</version>
</dependency>
</dependencies>
<build>
<resources>
<resource><directory>${web.build-dir}</directory></resource>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<skipAssembly>true</skipAssembly>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.10.1</version>
<configuration>
<release>17</release>
<showWarnings>true</showWarnings>
<showDeprecation>true</showDeprecation>
<forceJavacCompilerUse>true</forceJavacCompilerUse>
<annotationProcessors>
<!-- <annotationProcessor>com.google.auto.service.processor.AutoServiceProcessor</annotationProcessor> -->
<annotationProcessor>com.google.auto.value.processor.AutoValueProcessor</annotationProcessor>
</annotationProcessors>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<configuration>
<archive>
<manifestEntries>
<Graylog-Plugin-Properties-Path>${project.groupId}.${project.artifactId}</Graylog-Plugin-Properties-Path>
</manifestEntries>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<configuration>
<minimizeJar>false</minimizeJar>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<version>2.5.2</version>
<configuration>
<autoVersionSubmodules>true</autoVersionSubmodules>
<mavenExecutorId>forked-path</mavenExecutorId>
<tagNameFormat>@{project.version}</tagNameFormat>
<preparationGoals>clean test</preparationGoals>
<goals>package</goals>
</configuration>
</plugin>
<plugin>
<artifactId>jdeb</artifactId>
<groupId>org.vafer</groupId>
<version>1.4</version>
<configuration>
<deb>${project.build.directory}/${project.artifactId}-${project.version}.deb</deb>
<dataSet>
<data>
<src>${project.build.directory}/${project.build.finalName}.jar</src>
<type>file</type>
<mapper>
<type>perm</type>
<prefix>${graylog.plugin-dir}</prefix>
<filemode>644</filemode>
<user>root</user>
<group>root</group>
</mapper>
</data>
</dataSet>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>rpm-maven-plugin</artifactId>
<version>2.1.4</version>
<configuration>
<group>Application/Internet</group>
<prefixes>
<prefix>/usr</prefix>
</prefixes>
<defineStatements>
<defineStatement>_unpackaged_files_terminate_build 0</defineStatement>
<defineStatement>_binaries_in_noarch_packages_terminate_build 0</defineStatement>
</defineStatements>
<defaultFilemode>644</defaultFilemode>
<defaultDirmode>755</defaultDirmode>
<defaultUsername>root</defaultUsername>
<defaultGroupname>root</defaultGroupname>
<mappings>
<mapping>
<directory>${graylog.plugin-dir}</directory>
<sources>
<source>
<location>${project.build.directory}/</location>
<includes>
<include>${project.build.finalName}.jar</include>
</includes>
</source>
</sources>
</mapping>
</mappings>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>web-interface-build</id>
<activation>
<property>
<name>!skip.web.build</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>com.github.eirslett</groupId>
<artifactId>frontend-maven-plugin</artifactId>
<executions>
<execution>
<id>install node and yarn</id>
<goals>
<goal>install-node-and-yarn</goal>
</goals>
<configuration>
<nodeVersion>${nodejs.version}</nodeVersion>
<yarnVersion>${yarn.version}</yarnVersion>
</configuration>
</execution>
<execution>
<id>yarn install</id>
<goals>
<goal>yarn</goal>
</goals>
<!-- Optional configuration which provides for running any npm command -->
<configuration>
<arguments>install</arguments>
</configuration>
</execution>
<execution>
<id>yarn run build</id>
<goals>
<goal>yarn</goal>
</goals>
<configuration>
<arguments>run build</arguments>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

8
src/deb/control/control Normal file
View File

@ -0,0 +1,8 @@
Package: [[name]]
Version: [[version]]
Architecture: all
Maintainer: johan <johan@nosd.in>
Section: web
Priority: optional
Depends: graylog-server | graylog-radio
Description: [[description]]

View File

@ -0,0 +1,40 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
package in.nosd.redis;
import org.graylog2.plugin.Plugin;
import org.graylog2.plugin.PluginMetaData;
import org.graylog2.plugin.PluginModule;
import java.util.Collection;
import java.util.Collections;
/**
* This is the plugin. Your class should implement one of the existing plugin
* interfaces. (i.e. AlarmCallback, MessageInput, MessageOutput)
*/
public class RedisLookupPlugin implements Plugin {
@Override
public PluginMetaData metadata() {
return new RedisLookupPluginMetaData();
}
@Override
public Collection<PluginModule> modules () {
return Collections.<PluginModule>singletonList(new RedisLookupPluginModule());
}
}

View File

@ -0,0 +1,73 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
package in.nosd.redis;
import org.graylog2.plugin.PluginMetaData;
import org.graylog2.plugin.ServerStatus;
import org.graylog2.plugin.Version;
import java.net.URI;
import java.util.Collections;
import java.util.Set;
/**
* Implement the PluginMetaData interface here.
*/
public class RedisLookupPluginMetaData implements PluginMetaData {
private static final String PLUGIN_PROPERTIES = "in.nosd.redis.graylog-plugin-redis-lookup/graylog-plugin.properties";
@Override
public String getUniqueId() {
return "in.nosd.redis.RedisLookupPluginPlugin";
}
@Override
public String getName() {
return "RedisLookupPlugin";
}
@Override
public String getAuthor() {
return "johan <johan@nosd.in>";
}
@Override
public URI getURL() {
return URI.create("https://github.com/https://git.nosd.in/yo/graylog-plugin-redis-lookup");
}
@Override
public Version getVersion() {
return Version.fromPluginProperties(getClass(), PLUGIN_PROPERTIES, "version", Version.from(0, 0, 0, "unknown"));
}
@Override
public String getDescription() {
// TODO Insert correct plugin description
return "Redis database lookup & write functions for the Graylog Pipeline Processor";
}
@Override
public Version getRequiredVersion() {
return Version.fromPluginProperties(getClass(), PLUGIN_PROPERTIES, "graylog.version", Version.from(0, 0, 0, "unknown"));
}
@Override
public Set<ServerStatus.Capability> getRequiredCapabilities() {
return Collections.emptySet();
}
}

View File

@ -0,0 +1,87 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
package in.nosd.redis;
import com.google.inject.Binder;
import com.google.inject.TypeLiteral;
import com.google.inject.multibindings.MapBinder;
import org.graylog.plugins.pipelineprocessor.ast.functions.Function;
import org.graylog2.plugin.PluginConfigBean;
import org.graylog2.plugin.PluginModule;
import in.nosd.redis.dataadapters.RedisLookupDataAdapter;
import in.nosd.redis.functions.RedisLookupPluginFunction;
import java.util.Collections;
import java.util.Set;
/**
* Extend the PluginModule abstract class here to add you plugin to the system.
*/
public class RedisLookupPluginModule extends PluginModule {
/**
* Returns all configuration beans required by this plugin.
*
* Implementing this method is optional. The default method returns an empty {@link Set}.
*/
@Override
public Set<? extends PluginConfigBean> getConfigBeans() {
return Collections.emptySet();
}
@Override
protected void configure() {
/*
* Register your plugin types here.
*
* Examples:
*
* addMessageInput(Class<? extends MessageInput>);
* addMessageFilter(Class<? extends MessageFilter>);
* addMessageOutput(Class<? extends MessageOutput>);
* addPeriodical(Class<? extends Periodical>);
* addAlarmCallback(Class<? extends AlarmCallback>);
* addInitializer(Class<? extends Service>);
* addRestResource(Class<? extends PluginRestResource>);
*
*
* Add all configuration beans returned by getConfigBeans():
*
* addConfigBeans();
*/
addMessageProcessorFunction(RedisLookupPluginFunction.NAME, RedisLookupPluginFunction.class);
installLookupDataAdapter2(RedisLookupDataAdapter.NAME, RedisLookupDataAdapter.class,
RedisLookupDataAdapter.Factory.class, RedisLookupDataAdapter.Config.class);
addConfigBeans();
}
private void addMessageProcessorFunction(String name, Class<? extends Function<?>> functionClass) {
addMessageProcessorFunction(binder(), name, functionClass);
}
private MapBinder<String, Function<?>> processorFunctionBinder(Binder binder) {
return MapBinder.newMapBinder(binder, TypeLiteral.get(String.class), new TypeLiteral<Function<?>>() {});
}
private void addMessageProcessorFunction(Binder binder, String name, Class<? extends Function<?>> functionClass) {
processorFunctionBinder(binder).addBinding(name).to(functionClass);
}
}

View File

@ -0,0 +1,39 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
package in.nosd.redis;
import org.graylog2.plugin.Plugin;
import org.graylog2.plugin.PluginMetaData;
import org.graylog2.plugin.PluginModule;
import java.util.Collection;
import java.util.Collections;
/**
* Implement the Plugin interface here.
*/
public class RedisLookupPluginPlugin implements Plugin {
@Override
public PluginMetaData metadata() {
return new RedisLookupPluginMetaData();
}
@Override
public Collection<PluginModule> modules () {
return Collections.<PluginModule>singletonList(new RedisLookupPluginModule());
}
}

View File

@ -0,0 +1,288 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
package in.nosd.redis.dataadapters;
import com.codahale.metrics.Meter;
import com.codahale.metrics.MetricRegistry;
import com.codahale.metrics.Timer;
import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonTypeName;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.google.auto.value.AutoValue;
import org.graylog2.lookup.dto.DataAdapterDto;
import com.google.common.collect.ArrayListMultimap;
import com.google.common.collect.Multimap;
import com.google.inject.assistedinject.Assisted;
import org.apache.commons.lang3.StringUtils;
import io.lettuce.core.RedisClient;
import io.lettuce.core.RedisURI;
import io.lettuce.core.api.StatefulRedisConnection;
import io.lettuce.core.api.sync.RedisCommands;
/*
//To delete after clean
import okhttp3.HttpUrl;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.Response;
import okhttp3.ResponseBody;
//END To delete after clean
*/
import in.nosd.redis.dataadapters.AutoValue_RedisLookupDataAdapter_Config;
import org.graylog2.plugin.lookup.LookupCachePurge;
import org.graylog2.plugin.lookup.LookupDataAdapter;
import org.graylog2.plugin.lookup.LookupDataAdapterConfiguration;
import org.graylog2.plugin.lookup.LookupResult;
import org.joda.time.Duration;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.annotation.Nullable;
import com.google.inject.Inject;
import com.google.inject.assistedinject.Assisted;
import javax.validation.constraints.Min;
import javax.validation.constraints.NotEmpty;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.StringJoiner;
public class RedisLookupDataAdapter extends LookupDataAdapter {
private static final Logger LOG = LoggerFactory.getLogger(RedisLookupDataAdapter.class);
public static final String NAME = "RedisLookup";
private static final Duration REFRESH_INTERVAL_DURATION = Duration.ZERO;
private final Config config;
private final RedisClient client;
private RedisCommands<String, String> commands;
private final Timer redisGetRequestTimer;
private final Meter redisGetRequestErrors;
private final Timer redisSetRequestTimer;
private final Meter redisSetRequestErrors;
@Inject
public RedisLookupDataAdapter(@Assisted("dto") DataAdapterDto dto,
MetricRegistry metricRegistry) {
super(dto, metricRegistry);
this.config = (Config) dto.config();
RedisURI redisUri = RedisURI.Builder.redis(this.config.redisHost(),this.config.redisPort())
.withPort(this.config.redisPort())
//.withAuthentication(this.config.redisUsername(), this.config.redisPassword())
.withDatabase(this.config.redisDB())
.build();
this.client = RedisClient.create(redisUri);
/*this.client = RedisClient.create(RedisURI.Builder.redis(this.config.redisHost(),this.config.redisPort())
.withPort(this.config.redisPort())
.withAuthentication(this.config.redisUsername(), this.config.redisPassword())
.withDatabase(this.config.redisDB())
.build());
*/
this.redisGetRequestTimer = metricRegistry.timer(MetricRegistry.name(getClass(), "redisGetRequestTime"));
this.redisGetRequestErrors = metricRegistry.meter(MetricRegistry.name(getClass(), "redisGetRequestErrors"));
this.redisSetRequestTimer = metricRegistry.timer(MetricRegistry.name(getClass(), "redisSetRequestTime"));
this.redisSetRequestErrors = metricRegistry.meter(MetricRegistry.name(getClass(), "redisSetRequestErrors"));
}
// Add code to initialise Redis connection
@Override
protected void doStart() throws Exception {
StatefulRedisConnection<String, String> connection = this.client.connect();
this.commands = connection.sync();
}
// Add code to close Redis connection
@Override
protected void doStop() throws Exception {
}
@Override
public Duration refreshInterval() {
return REFRESH_INTERVAL_DURATION;
}
@Override
protected void doRefresh(LookupCachePurge cachePurge) throws Exception {
doStart();
cachePurge.purgeAll();
}
@Override
protected LookupResult doGet(Object key) {
final Timer.Context time = redisGetRequestTimer.time();
final String trimmedKey = StringUtils.trimToNull(key.toString());
if (trimmedKey == null) {
LOG.debug("A blank key was supplied");
return getEmptyResult();
}
try {
final String value = this.commands.get(trimmedKey);
if (value == null) {
LOG.warn("Redis GET request for key <{}> returned null, key do not exists.", trimmedKey);
redisGetRequestErrors.mark();
return LookupResult.empty();
}
return LookupResult.single(value);
//return LookupResult.single("coincoin");
} catch (Exception e) {
LOG.error("Redis GET request error for key <{}>", trimmedKey, e);
redisGetRequestErrors.mark();
return LookupResult.empty();
} finally {
time.stop();
}
}
@Override
public void set(Object key, Object value) {
//throw new UnsupportedOperationException();
LOG.warn("Entering Redis set function with {}={}", key.toString(), value.toString());
final Timer.Context time = redisSetRequestTimer.time();
try {
final String result = this.commands.set(key.toString(), value.toString());
if (!result.equals("OK")) {
LOG.warn("Redis SET key <{}> to value <{}> returned {}", key, value, result);
redisSetRequestErrors.mark();
return;
}
return;
} catch (Exception e) {
LOG.error("Redis SET key <{}> to value <{}> returned an exception: {}", key, value, e);
redisSetRequestErrors.mark();
return;
} finally {
time.stop();
}
}
public interface Factory extends LookupDataAdapter.Factory2<RedisLookupDataAdapter> {
@Override
RedisLookupDataAdapter create(@Assisted("dto") DataAdapterDto dto);
@Override
Descriptor getDescriptor();
}
public static class Descriptor extends LookupDataAdapter.Descriptor<Config> {
public Descriptor() {
super(NAME, Config.class);
}
@Override
public Config defaultConfiguration() {
return Config.builder()
.type(NAME)
.redisHost("127.0.0.1")
.redisPort(6379)
.redisDB(0)
/*.redisUsername("")
.redisPassword("")*/
.build();
}
}
@AutoValue
@JsonAutoDetect
@JsonDeserialize(builder = RedisLookupDataAdapter.Config.Builder.class)
@JsonTypeName(NAME)
@JsonInclude(JsonInclude.Include.NON_EMPTY)
public abstract static class Config implements LookupDataAdapterConfiguration {
@Override
@JsonProperty(TYPE_FIELD)
public abstract String type();
@JsonProperty("redis_host")
@NotEmpty
public abstract String redisHost();
@JsonProperty("redis_port")
@Min(1)
public abstract int redisPort();
@JsonProperty("redis_database")
@Min(0)
public abstract int redisDB();
/*@JsonProperty("redis_username")
@Nullable
public abstract String redisUsername();
@JsonProperty("redis_password")
@Nullable
public abstract String redisPassword();*/
public static Builder builder() {
return new AutoValue_RedisLookupDataAdapter_Config.Builder();
}
/* @Override
public Optional<Multimap<String, String>> validate() {
final ArrayListMultimap<String, String> errors = ArrayListMultimap.create();
if (redisPort() < 1 || redisPort() > 65535) {
errors.put("redis_port", "Value cannot neither be smaller than 1 nor greater than 65535");
}
if (redisDB() < 0) {
errors.put("redis_database", "Value cannot be smaller than 0");
}
return errors.isEmpty() ? Optional.empty() : Optional.of(errors);
}
*/
@AutoValue.Builder
public abstract static class Builder {
@JsonCreator
public static Builder create() {
return Config.builder();
}
@JsonProperty(TYPE_FIELD)
public abstract Builder type(String type);
@JsonProperty("redis_host")
public abstract Builder redisHost(String redisHost);
@JsonProperty("redis_port")
public abstract Builder redisPort(int redisPort);
@JsonProperty("redis_database")
public abstract Builder redisDB(int redisDB);
/*@JsonProperty("redis_username")
public abstract Builder redisUsername(String redisUsername);
@JsonProperty("redis_password")
public abstract Builder redisPassword(String redisPassword);*/
public abstract Config build();
}
}
}

View File

@ -0,0 +1,70 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
package in.nosd.redis.functions;
import com.google.common.collect.ForwardingMap;
import com.google.common.collect.ImmutableMap;
import java.util.Map;
public class GenericLookupResult extends ForwardingMap<String, Object> {
public static final String RESULTS_KEY = "threat_indicated";
private final ImmutableMap<String, Object> results;
public static final GenericLookupResult FALSE = new FalseGenericLookupResult();
public static final GenericLookupResult TRUE = new TrueGenericLookupResult();
private GenericLookupResult(ImmutableMap<String, Object> fields) {
this.results = fields;
}
public Map<String, Object> getResults() {
return results;
}
public boolean isMatch() {
return ((boolean) getResults().get(RESULTS_KEY));
}
@Override
protected Map<String, Object> delegate() {
return getResults();
}
private static class FalseGenericLookupResult extends GenericLookupResult {
private static final ImmutableMap<String, Object> FALSE = ImmutableMap.<String, Object>builder()
.put(RESULTS_KEY, false)
.build();
private FalseGenericLookupResult() {
super(FALSE);
}
}
private static class TrueGenericLookupResult extends GenericLookupResult {
private static final ImmutableMap<String, Object> TRUE = ImmutableMap.<String, Object>builder()
.put(RESULTS_KEY, true)
.build();
private TrueGenericLookupResult() {
super(TRUE);
}
}
}

View File

@ -0,0 +1,22 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
package in.nosd.redis.functions;
import org.graylog.plugins.pipelineprocessor.ast.functions.AbstractFunction;
public abstract class LookupTableFunction<R> extends AbstractFunction<R> {
}

View File

@ -0,0 +1,85 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
package in.nosd.redis.functions;
import com.google.common.base.Strings;
import in.nosd.redis.functions.GenericLookupResult;
import in.nosd.redis.functions.LookupTableFunction;
import org.graylog.plugins.pipelineprocessor.EvaluationContext;
import org.graylog.plugins.pipelineprocessor.ast.functions.FunctionArgs;
import org.graylog.plugins.pipelineprocessor.ast.functions.FunctionDescriptor;
import org.graylog.plugins.pipelineprocessor.ast.functions.ParameterDescriptor;
import org.graylog2.lookup.LookupTableService;
import org.graylog2.plugin.lookup.LookupResult;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.inject.Inject;
public class RedisLookupPluginFunction extends LookupTableFunction<GenericLookupResult> {
private static final Logger LOG = LoggerFactory.getLogger(RedisLookupPluginFunction.class);
public static final String NAME = "redis_lookup";
private static final String VALUE = "value";
private static final String LOOKUP_TABLE_NAME = "redis-lookup";
private final ParameterDescriptor<String, String> keyParam = ParameterDescriptor.string(VALUE).description("The key to look up.").build();
private final LookupTableService.Function lookupFunction;
@Inject
public RedisLookupPluginFunction(final LookupTableService lookupTableService) {
this.lookupFunction = lookupTableService.newBuilder().lookupTable(LOOKUP_TABLE_NAME).build();
}
@Override
public GenericLookupResult evaluate(FunctionArgs args, EvaluationContext context) {
String key = keyParam.required(args, context);
if (key == null) {
LOG.error("NULL parameter passed to Redis lookup.");
return null;
}
LOG.debug("Running Redis lookup for key [{}].", key);
final LookupResult lookupResult = this.lookupFunction.lookup(key.trim());
if (lookupResult != null && !lookupResult.isEmpty()) {
// If not a String, then fall through to false at the end of the method.
final Object value = lookupResult.singleValue();
if (value instanceof String) {
return !Strings.isNullOrEmpty((String) value) ? GenericLookupResult.TRUE : GenericLookupResult.FALSE;
}
}
return GenericLookupResult.FALSE;
}
@Override
public FunctionDescriptor<GenericLookupResult> descriptor() {
return FunctionDescriptor.<GenericLookupResult>builder()
.name(NAME)
.description("Match a key into Redis instance and return value")
.params(keyParam)
.returnType(GenericLookupResult.class)
.build();
}
}

View File

@ -0,0 +1 @@
in.nosd.redis.RedisLookupPluginPlugin

View File

@ -0,0 +1,12 @@
# The plugin version
version=${project.version}
# The required Graylog server version
graylog.version=${graylog.version}
# When set to true (the default) the plugin gets a separate class loader
# when loading the plugin. When set to false, the plugin shares a class loader
# with other plugins that have isolated=false.
#
# Do not disable this unless this plugin depends on another plugin!
isolated=true

View File

@ -0,0 +1,34 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
/* eslint-disable react/no-unescaped-entities, no-template-curly-in-string */
import React from 'react';
class RedisLookupAdapterDocumentation extends React.Component {
render() {
const style = { marginBottom: 10 };
return (
<div>
<p style={style}>
The Redis Lookup data adapter lookup redis for the given key and returns the values .
</p>
</div>
)
;
}
}
export default RedisLookupAdapterDocumentation;

View File

@ -0,0 +1,85 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
import React, { useCallback, useEffect, useRef, useState } from 'react';
import PropTypes from 'prop-types';
//import { Button } from 'components/graylog';
import { Input } from 'components/bootstrap';
class RedisLookupAdapterFieldSet extends React.Component {
static propTypes = {
config: PropTypes.shape({
redis_host: PropTypes.string.isRequired,
}).isRequired,
updateConfig: PropTypes.func.isRequired,
handleFormEvent: PropTypes.func.isRequired,
validationState: PropTypes.func.isRequired,
validationMessage: PropTypes.func.isRequired,
};
handleSelect = (fieldName) => {
return (selectedIndicator) => {
const config = lodash.cloneDeep(this.props.config);
config[fieldName] = selectedIndicator;
this.props.updateConfig(config);
};
};
render() {
const { config } = this.props;
return (
<fieldset>
<Input type="text"
id="redis_host"
name="redis_host"
label="Redis host"
required
onChange={this.props.handleFormEvent}
help={this.props.validationMessage('redis_host', 'Your Redis Host')}
bsStyle={this.props.validationState('redis_host')}
value={config.redis_host}
labelClassName="col-sm-3"
wrapperClassName="col-sm-9" />
<Input type="text"
id="redis_port"
name="redis_port"
label="Redis port"
required
onChange={this.props.handleFormEvent}
help={this.props.validationMessage('redis_port', 'Redis port instance is listening on')}
bsStyle={this.props.validationState('redis_port')}
value={config.redis_port}
labelClassName="col-sm-3"
wrapperClassName="col-sm-9" />
<Input type="text"
id="redis_database"
name="redis_database"
label="Redis database"
required
onChange={this.props.handleFormEvent}
help={this.props.validationMessage('redis_database', 'Redis database')}
bsStyle={this.props.validationState('redis_database')}
value={config.redis_database}
labelClassName="col-sm-3"
wrapperClassName="col-sm-9" />
</fieldset>
);
}
}
export default RedisLookupAdapterFieldSet;

View File

@ -0,0 +1,42 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
// eslint-disable-next-line react/prefer-stateless-function
import React from 'react';
import PropTypes from 'prop-types';
class RedisLookupAdapterSummary extends React.Component {
static propTypes = {
dataAdapter: PropTypes.shape({
config: PropTypes.shape({
redis_host: PropTypes.string.isRequired,
}),
}),
};
render() {
const { config } = this.props.dataAdapter;
return (
<dl>
<dt>Redis host</dt>
<dd>{config.redis_host || 'n/a'}</dd>
</dl>
);
}
}
export default RedisLookupAdapterSummary;

50
src/web/index.jsx Normal file
View File

@ -0,0 +1,50 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
// eslint-disable-next-line no-unused-vars
import webpackEntry from 'webpack-entry';
import { PluginManifest, PluginStore } from 'graylog-web-plugin/plugin';
import packageJson from '../../package.json';
//import StoreProvider from 'injection/StoreProvider';
// https://github.com/Graylog2/graylog2-server/blob/master/graylog2-web-interface/src/stores/plugins/PluginsStore.js ?
//const PStore = StoreProvider.getStore('core.Plugins')
debugger;
console.error('Before registering RedisLookup type lookupTableAdapters');
import RedisLookupAdapterDocumentation from './dataadapters/RedisLookupAdapterDocumentation';
import RedisLookupAdapterFieldSet from './dataadapters/RedisLookupAdapterFieldSet';
import RedisLookupAdapterSummary from './dataadapters/RedisLookupAdapterSummary';
const manifest = new PluginManifest(packageJson, {
lookupTableAdapters: [
{
type: 'RedisLookup',
displayName: 'Redis Lookup',
formComponent: RedisLookupAdapterFieldSet,
summaryComponent: RedisLookupAdapterSummary,
documentationComponent: RedisLookupAdapterDocumentation,
},
],
});
debugger;
console.error('Registering RedisLookup type lookupTableAdapters');
PluginStore.register(manifest);

42
src/web/index.jsx.raw Normal file
View File

@ -0,0 +1,42 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
import 'webpack-entry';
import { PluginManifest, PluginStore } from 'graylog-web-plugin/plugin';
import packageJson from '../../package.json';
const manifest = new PluginManifest(packageJson, {
/* This is the place where you define which entities you are providing to the web interface.
Right now you can add routes and navigation elements to it.
Examples: */
// Adding a route to /sample, rendering YourReactComponent when called:
// routes: [
// { path: '/sample', component: YourReactComponent, permissions: 'inputs:create' },
// ],
// Adding an element to the top navigation pointing to /sample named "Sample":
// navigation: [
// { path: '/sample', description: 'Sample' },
// ]
});
PluginStore.register(manifest);

24
webpack.config.js Normal file
View File

@ -0,0 +1,24 @@
/*
* Copyright (C) 2020 Graylog, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the Server Side Public License, version 1,
* as published by MongoDB, Inc.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* Server Side Public License for more details.
*
* You should have received a copy of the Server Side Public License
* along with this program. If not, see
* <http://www.mongodb.com/licensing/server-side-public-license>.
*/
const path = require('path');
const { PluginWebpackConfig } = require('graylog-web-plugin');
const { loadBuildConfig } = require('graylog-web-plugin');
// Remember to use the same name here and in `getUniqueId()` in the java MetaData class
module.exports = new PluginWebpackConfig(__dirname, 'in.nosd.redis.RedisLookupPluginPlugin', loadBuildConfig(path.resolve(__dirname, './build.config')), {
// Here goes your additional webpack configuration.
});

8715
yarn.lock Normal file

File diff suppressed because it is too large Load Diff