Obtenha Kibana do repositório Github
sudo su
cd /srv
git clone --branch=kibana-ruby https://github.com/rashidkpc/Kibana.git
mv Kibana kibana
cd kibana
bundle install
ln -s static public # we will need it to run kibana under nginx and passenger
Kibana também precisa do Elasticsearch (por favor, não instale o Elasticsearch mais recente – ele não é compatível com a versão atual do Logstash)
cd ~
wget https://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.19.10.deb
dpkg -i elasticsearch-0.19.10.deb
rm elasticsearch-0.19.10.deb
Configuração do Elasticsearch (arquivo /etc/elasticsearch/elasticsearch.yaml):
cluster.name: elasticsearch
node.name: "your_host_name"
node.master: true
node.data: true
index.number_of_shards: 5
index.number_of_replicas: 1
index.number_of_shards: 1
index.number_of_replicas: 0
bootstrap.mlockall: true
network.host: 127.0.0.1
http.max_content_length: 1000mb
index.search.slowlog.level: TRACE
index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 5s
index.search.slowlog.threshold.query.debug: 2s
index.search.slowlog.threshold.query.trace: 500ms
index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 800ms
index.search.slowlog.threshold.fetch.debug: 500ms
index.search.slowlog.threshold.fetch.trace: 200ms
monitor.jvm.gc.ParNew.warn: 1000ms
monitor.jvm.gc.ParNew.info: 700ms
monitor.jvm.gc.ParNew.debug: 400ms
monitor.jvm.gc.ConcurrentMarkSweep.warn: 10s
monitor.jvm.gc.ConcurrentMarkSweep.info: 5s
monitor.jvm.gc.ConcurrentMarkSweep.debug: 2s
Configuração básica do Kibana (arquivo /srv/kibana/KibanaConfig.rb):
module KibanaConfig
# A Note: While the only option you really have to set is "Elasticsearch" it
# is HIGHLY recommended you glance over every option. I personally consider
# 'Facet_index_limit' really important.
# Your elastic search server(s). This may be set as an array for round robin
# load balancing
# Elasticsearch = ["elasticsearch1:9200","elasticsearch2:9200"]
Elasticsearch = "localhost:9200"
#Set the Net::HTTP read/open timeouts for the connection to the ES backend
ElasticsearchTimeout = 500
# The port Kibana should listen on
KibanaPort = 8998
# The adress ip Kibana should listen on. Comment out or set to
# 0.0.0.0 to listen on all interfaces.
KibanaHost = '0.0.0.0'
# The record type as defined in your logstash configuration.
# Seperate multiple types with a comma, no spaces. Leave blank
# for all.
Type = ''
# Results to show per page
Per_page = 50
# Timezone. Leave this set to 'user' to have the user's browser autocorrect.
# Otherwise, set a timezone string
# Examples: 'UTC', 'America/Phoenix', 'Europe/Athens', MST
# You can use `date +%Z` on linux to get your timezone string
Timezone = 'user'
# Format for timestamps. Defaults to mm/dd HH:MM:ss.
# For syntax see: http://blog.stevenlevithan.com/archives/date-time-format
# Time_format = 'isoDateTime'
Time_format = 'mm/dd HH:MM:ss'
# Change which fields are shown by default. Must be set as an array
# Default_fields = ['@fields.vhost','@fields.response','@fields.request']
Default_fields = ['@message']
# The default operator used if no explicit operator is specified.
# For example, with a default operator of OR, the query capital of
# Hungary is translated to capital OR of OR Hungary, and with default
# operator of AND, the same query is translated to capital AND of AND
# Hungary. The default value is OR.
Default_operator = 'OR'
# When using analyze, use this many of the most recent
# results for user's query
Analyze_limit = 2000
# Show this many results in analyze/trend/terms/stats modes
Analyze_show = 25
# Show this many results in an rss feed
Rss_show = 25
# Show this many results in an exported file
Export_show = 2000
# Delimit exported file fields with what?
# You may want to change this to something like "t" (tab) if you have
# commas in your logs
Export_delimiter = ","
# You may wish to insert a default search which all user searches
# must match. For example @source_host:www1 might only show results
# from www1.
Filter = ''
# When searching, Kibana will attempt to only search indices
# that match your timeframe, to make searches faster. You can
# turn this behavior off if you use something other than daily
# indexing
Smart_index = true
# You can define your custom pattern here for index names if you
# use something other than daily indexing. Pattern needs to have
# date formatting like '%Y.%m.%d'. Will accept an array of smart
# indexes.
# Smart_index_pattern = ['logstash-web-%Y.%m.%d', 'logstash-mail-%Y.%m.%d']
Smart_index_pattern = 'logstash-%Y.%m.%d'
# Number of seconds between each index. 86400 = 1 day.
Smart_index_step = 86400
# ElasticSearch has a default limit on URL size for REST calls,
# so Kibana will fall back to _all if a search spans too many
# indices. Use this to set that 'too many' number. By default this
# is set really high, ES might not like this
Smart_index_limit = 150
# Elasticsearch has an internal mechanism called "faceting" for performing
# analysis that we use for the "Stats" and "Terms" modes. However, on large
# data sets/queries facetting can cause ES to crash if there isn't enough
# memory available. It is suggested that you limit the number of indices that
# Kibana will use for the "Stats" and "Terms" to prevent ES crashes. For very
# large data sets and undersized ES clusers, a limit of 1 is not unreasonable.
# Default is 0 (unlimited)
Facet_index_limit = 0
# You probably don't want to touch anything below this line
# unless you really know what you're doing
# Primary field. By default Elastic Search has a special
# field called _all that is searched when no field is specified.
# Dropping _all can reduce index size significantly. If you do that
# you'll need to change primary_field to be '@message'
Primary_field = '_all'
# Default Elastic Search index to query
Default_index = '_all'
# TODO: This isn't functional yet
# Prevent wildcard search terms which result in extremely slow queries
# See: http:#www.elasticsearch.org/guide/reference/query-dsl/wildcard-query.html
Disable_fullscan = false
# Set headers to allow kibana to be loaded in an iframe from a different origin.
Allow_iframed = false
# Use this interval as fallback.
Fallback_interval = 900
end
Instale o Logstash
sudo su
apt-get install openjdk-7-jre
mkdir /etc/logstash
cd /etc/logstash
wget https://logstash.objects.dreamhost.com/release/logstash-1.1.5-monolithic.jar -O logstash.jar
mkdir /var/log/logstash
Configure o Logstash (arquivo /etc/logstash/logstash.conf):
input {
file {
type => nginx_web
path => ["/var/log/nginx/*"]
exclude => ["*.gz"]
sincedb_path => "$HOME/.sincedb"
}
}
input {
file {
type => "rails_app"
path => [ "/var/log/rails_app*.log" ]
exclude => ["*.gz"]
sincedb_path => "$HOME/.sincedb"
}
}
output {
elasticsearch {
host => "127.0.0.1"
port => 9300
}
}
Arquivo de inicialização para Logstash:
#! /bin/sh
### BEGIN INIT INFO
# Provides: logstash-shipper
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start daemon at boot time
# Description: Enable service provided by daemon.
### END INIT INFO
. /lib/lsb/init-functions
name="logstash"
logstash_bin="/usr/bin/java -- -jar /etc/logstash/logstash.jar"
logstash_conf="/etc/logstash/logstash.conf"
logstash_log="/var/log/logstash/logstash.log"
pid_file="/var/run/$name.pid"
start () {
command="${logstash_bin} agent -f $logstash_conf --log ${logstash_log}"
log_daemon_msg "Starting $name"
if start-stop-daemon --start --quiet --oknodo --pidfile "$pid_file" -b -m --exec $command; then
log_end_msg 0
else
log_end_msg 1
fi
}
stop () {
start-stop-daemon --stop --quiet --oknodo --pidfile "$pid_file"
}
status () {
status_of_proc -p $pid_file "" "$name"
}
case $1 in
start)
if status; then exit 0; fi
start
;;
stop)
stop
;;
reload)
stop
start
;;
restart)
stop
start
;;
status)
status && exit 0 || exit $?
;;
*)
echo "Usage: $0 {start|stop|restart|reload|status}"
exit 1
;;
esac
exit 0
Você também deve adicionar permissão para executar o arquivo e iniciar o processo Logstash:
$ chmod +x /etc/init.d/logstash
$ /etc/init.d/logstash start
Configuração do Nginx para Kibana (arquivo /etc/nginx/nginx.conf):
worker_processes 2;
events {
worker_connections 1024;
}
http {
passenger_root /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.18;
passenger_ruby /usr/local/bin/ruby;
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'$ssl_cipher $request_time $host';
sendfile on;
keepalive_timeout 65;
gzip on;
server {
listen 80;
server_name kibana;
root /srv/kibana/public;
passenger_enabled on;
rack_env production;
}
}
Configure o logger para o aplicativo Rails:
* instale o SyslogLogger, apenas adicione a seguinte linha ao Gemfile
gem "SyslogLogger", "~> 2.0", :require => 'syslog/logger'
logger de configuração em seu aplicativo, em production.rb
config.logger = Syslog :: Logger.new ” nome do seu aplicativo _ # {Rails.env}”
Aproveitar!