filter {
if "jdbc" in [tags] {
mutate {
convert => {
"id_sync" => "text"
if ["aut", "log", "lan", "ltr"] in [tags] {
ruby {
code => '
fieldname = event.get("type") + "datref"
event.set("datref", event.get(fieldname))
ERROR_LOG:
[2019-01-23T15:02:20,176][ERROR][logstash.agent]
Failed to execute action {:action=>LogStash::PipelineAction::Stop/pipeline_id:basa-banklink, :exception=>"NoMethodError",
:message=>"undefined method `unlock' for nil:NilClass",
:backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/plugin_mixins/jdbc/jdbc.rb:213:in
`close_jdbc_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.13/lib/logstash/inputs/jdbc.rb:268:in `stop'",
"/usr/share/logstash/logstash-core/lib/logstash/inputs/base.rb:87:in `do_stop'", "org/jruby/RubyArray.java:1734:in `each'",
"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:464:in `stop_inputs'",
"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:447:in `shutdown'",
"/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/stop.rb:14:in
`block in execute'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/stop.rb:13:in `execute'",
"/usr/share/logstash/logstash-core/lib/logstash/agent.rb:317:in `block in converge_state'"]}
That log file does not contain that error. Please post a link to one that does.
Your configuration has 8 jdbc filters. My guess is that something is causing one of them to be closed or restarted before it has been initialialized. In fact it fails to load the pipeline because of
[ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"basa-banklink", :plugin=>"#<LogStash::FilterDelegator:0x4c87dc67 @metric_events_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @metric_events_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @metric_events_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration_in_millis value:0, @id="896d1d111b0ae51e2f2b01f7019e66b763cacdce97dfeb0b072b90fd00d33dea", @klass=LogStash::Filters::Mutate, @metric_events=#LogStash::Instrument::NamespacedMetric:0x6e1186f9, @filter=<LogStash::Filters::Mutate convert=>{"id_sync"=>"text"}, id=>"896d1d111b0ae51e2f2b01f7019e66b763cacdce97dfeb0b072b90fd00d33dea", enable_metric=>true, periodic_flush=>false>>", :error=>"translation missing: en.logstash.agent.configuration.invalid_plugin_register", :thread=>"#<Thread:0xe83bbea@/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:51 run>"}
I wonder if a pipeline restart after that would cause the problem.
Sorry for the previous log.
Here is the log containing the error: new_error_log.log
at the same address previously reported.
I reduced the configuration, I commented the pipeline.id: default
What I could not understand is that if I leave only the ruby flag code inside the filter, logstash starts and works without problem.
What an excellent bug! text is not valid there. You want to do a convert to 'string'. Because the mutate filter cannot initialize the rest of the pipeline configuration does not happen and it never initializes the 8 jdbc inputs. So when the config is re-read causing a pipeline restart or in the case of a shutdown you get an error referencing the unlock.
Thanks for the feedback. I was writing the text below when you answered me.
I changed the amount of JDBC entries from 8 to 1, but I got the same error.
I believe it's something with the mutate filter, because if I remove the block exactly:
mutate {
convert => {
"id_sync" => "text"
Logstash starts successfully.
So I'll rethink how I'll do this conversion out of logstash. Would you have any suggestions?
Do I need to open the bug somewhere or do you already do this just by having the post?
That I hesitate, I was paying attention to the types already indexed in elasticsearch and not to the types described in the address: Mutate filter plugin | Logstash Reference [8.11] | Elastic
Sorry for the unnecessary post.
Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S.
and in other countries
Brand
Code of Conduct
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.