Scribe安裝使用

摘要： 前言 Scribe是Facebook開源的日誌收集工具，我建議大家還是老老實實使用ELK或者Splunk，不要瞎折騰Scribe Centos安裝Scribe真是累 更新歷史 2018年09月27日 - 初稿 閱讀原文 - ...

前言

Scribe是Facebook開源的日誌收集工具，我建議大家還是老老實實使用ELK或者Splunk，不要瞎折騰Scribe

Centos安裝Scribe真是累

更新歷史

2018年09月27日 - 初稿

閱讀原文 -ofollow,noindex"> https:// wsgzao.github.io/post/s cribe/

擴充套件閱讀

Scribe - https:// github.com/facebookarch ive/scribe Thrift - http:// thrift.apache.org/docs/ install/centos

Scribe簡介

Scribe is a server for aggregating log data that's streamed in real time from clients. It is designed to be scalable and reliable.

See the Scribe Wiki for documentation: http:// wiki.github.com/faceboo k/scribe

install scribe

OS: CentOS 7.x

This guide shows how to install the precompiled scribed binary and its dependencies on a new server. Care must be taken to use the required versions forboost (1.44.0) ,thrift (0.7.0) andlibevent (1.4.10) .

# install necessary tools
sudo yum install git make flex bison libtool automake openssl-devel libevent libevent-devel python-devel gcc-c++ byacc java-1.7.0-openjdk ant

# update autoconf
cd ~
sudo rpm -e --nodeps `rpm -qf /usr/bin/autoconf`
wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
tar xzf autoconf-2.69.tar.gz
cd autoconf-2.69/
./configure
make
sudo make install
# make sure /usr/local/bin is in system PATH: add following line to ~/.bashrc
vi ~/.bashrc
export PATH=$PATH:/usr/local/bin

# install boost
cd ~
wget http://jaist.dl.sourceforge.net/project/boost/boost/1.44.0/boost_1_44_0.tar.gz
tar xzf boost_1_44_0.tar.gz
cd boost_1_44_0/
./bootstrap.sh
sudo ./bjam install

# install thrift
cd ~
git clone https://github.com/apache/thrift.git
cd thrift/
git fetch
git branch -a
git checkout 0.7.x
./bootstrap.sh
./configure
make
cp compiler/cpp/thrifty.hh compiler/cpp/thrifty.h
make
sudo make install
cd lib/py/
sudo python setup.py install

# install fb303
cd ~/thrift/contrib/fb303/
./bootstrap.sh
./configure CPPFLAGS="-DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H"
make
sudo make install
cd py/
sudo python setup.py install

# install libevent
cd ~
wget https://github.com/downloads/libevent/libevent/libevent-1.4.10-stable.tar.gz
tar xf libevent-1.4.10-stable.tar.gz 
cd libevent-1.4.10-stable
./configure
make
make install

# Copy the scribed binary and put it in the desired directory
scp scribe.tar.gz 10.65.32.134:/tmp
scp scribed 10.65.32.134:/tm
# unzip and mkdir
mkdir /opt/running
cd /opt/running
cp /tmp/scribe.tar.gz .
tar xzvf scribe.tar.gz
rm -f scribe.tar.gz
mkdir -p /data/gop/live/primary/gop
mkdir -p /data/gop/live/secondary/gop
cp /tmp/scribed /usr/local/bin/scribed
cd scribe/

[root@vm4 scribe]# cat run_scribed.sh 
#!/bin/bash

while true
do
export LD_LIBRARY_PATH=/usr/lib:/usr/local/lib
/usr/local/bin/scribed scribe_11315.conf 1>output/scribe.log 2>output/scribe.err
sleep 1
done

nohup sh run_scribed.sh &

[root@sg-gop-10-65-32-134 scribe]# netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local AddressForeign AddressStatePID/Program name
tcp00 0.0.0.0:100500.0.0.0:*LISTEN55310/zabbix_agentd
tcp00 0.0.0.0:384220.0.0.0:*LISTEN52781/sshd
tcp00 0.0.0.0:220.0.0.0:*LISTEN1305/sshd
tcp600 :::10050:::*LISTEN55310/zabbix_agentd
tcp600 :::11315:::*LISTEN4294/scribed
tcp600 :::38422:::*LISTEN52781/sshd
tcp600 :::22:::*LISTEN1305/sshd