.NetCore實踐篇:分散式監控Zipkin持久化之殤
前言
本系列已寫了四篇文章,讀本篇之前,可以先讀前面幾篇。
思考大綱:ofollow,noindex">.Net架構篇:思考如何設計一款實用的分散式監控系統?
實踐篇一:.NetCore實踐篇:分散式監控客戶端ZipkinTracer從入門到放棄之路
實踐篇二:.NetCore實踐篇:分散式監控系統zipkin踩坑之路(二)
實踐篇三:.NetCore實踐篇:成功解決分散式監控ZipKin聚合依賴問題(三)
簡要回顧
zipkin
Zipkin是一種分散式跟蹤系統。它有助於收集解決微服務架構中的延遲問題所需的時序資料
zipkin官網zipkin4Net
zipkin4net是.NET客戶端庫。
zipkin4netzipkin-dependencies
這是一個Spark作業,它將從您的資料儲存區收集跨度,分析服務之間的連結,並存儲它們以供以後在Web UI中 呈現。
使用方法
如果記憶體不足時,java後跟上-Xmx1024m -Xms1024m引數,JAVA_OPTS的一些引數可參考Oracle官方說明配置預設JVM和Java引數
# ex to run the job to process yesterday's traces on OS/X $ STORAGE_TYPE=cassandra3 java -jar zipkin-dependencies.jar `date -uv-1d +%F` # or on Linux $ STORAGE_TYPE=cassandra3 java -jar zipkin-dependencies.jar `date -u -d '1 day ago' +%F`
SQL/">MySQL 儲存
* `MYSQL_DB`: 使用的資料庫,預設是 "zipkin". * `MYSQL_USER` and `MYSQL_PASS`: MySQL授權, 預設是空. * `MYSQL_HOST`: 預設主機(域名/ip)是localhost * `MYSQL_TCP_PORT`: 預設埠是 3306 * `MYSQL_USE_SSL`: 驗證 `javax.net.ssl.trustStore` 和 `javax.net.ssl.trustStorePassword`,預設不驗證。
示例
$ STORAGE_TYPE=mysql MYSQL_USER=root java -jar zipkin-dependencies.jar
詳情參考:
zipkin-dependencies實踐
建立使用資料庫
mysql> create database mytestdb; Query OK, 1 row affected (0.01 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | mytestdb | | performance_schema | | sys | | ttt | +--------------------+ 7 rows in set (0.00 sec) mysql> use mytestdb Database changed
使用sql語句建立zipkin表
CREATETABLE IF NOT EXISTS zipkin_spans ( `trace_id_high` BIGINT NOT NULL DEFAULT 0COMMENT 'If non zero, this means the trace uses 128 bit traceIds instead of 64bit', `trace_id` BIGINT NOT NULL, `id` BIGINT NOT NULL, `name` VARCHAR(255) NOT NULL, `parent_id` BIGINT, `debug` BIT(1), `start_ts` BIGINT COMMENT 'Span.timestamp():epoch micros used for endTs query and to implement TTL', `duration` BIGINT COMMENT 'Span.duration():micros used for minDuration and maxDuration query' )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci; ALTERTABLE zipkin_spans ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `id`) COMMENT'ignore insert on duplicate'; ALTERTABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`, `id`) COMMENT 'forjoining with zipkin_annotations'; ALTERTABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`) COMMENT 'forgetTracesByIds'; ALTERTABLE zipkin_spans ADD INDEX(`name`) COMMENT 'for getTraces and getSpanNames'; ALTERTABLE zipkin_spans ADD INDEX(`start_ts`) COMMENT 'for getTraces ordering andrange'; CREATETABLE IF NOT EXISTS zipkin_annotations ( `trace_id_high` BIGINT NOT NULL DEFAULT 0COMMENT 'If non zero, this means the trace uses 128 bit traceIds instead of 64bit', `trace_id` BIGINT NOT NULL COMMENT 'coincideswith zipkin_spans.trace_id', `span_id` BIGINT NOT NULL COMMENT 'coincideswith zipkin_spans.id', `a_key` VARCHAR(255) NOT NULL COMMENT'BinaryAnnotation.key or Annotation.value if type == -1', `a_value` BLOB COMMENT'BinaryAnnotation.value(), which must be smaller than 64KB', `a_type` INT NOT NULL COMMENT'BinaryAnnotation.type() or -1 if Annotation', `a_timestamp` BIGINT COMMENT 'Used toimplement TTL; Annotation.timestamp or zipkin_spans.timestamp', `endpoint_ipv4` INT COMMENT 'Null whenBinary/Annotation.endpoint is null', `endpoint_ipv6` BINARY(16) COMMENT 'Null whenBinary/Annotation.endpoint is null, or no IPv6 address', `endpoint_port` SMALLINT COMMENT 'Null whenBinary/Annotation.endpoint is null', `endpoint_service_name` VARCHAR(255) COMMENT'Null when Binary/Annotation.endpoint is null' )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci; ALTERTABLE zipkin_annotations ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `span_id`,`a_key`, `a_timestamp`) COMMENT 'Ignore insert on duplicate'; ALTERTABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`, `span_id`)COMMENT 'for joining with zipkin_spans'; ALTERTABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`) COMMENT 'forgetTraces/ByIds'; ALTERTABLE zipkin_annotations ADD INDEX(`endpoint_service_name`) COMMENT 'forgetTraces and getServiceNames'; ALTERTABLE zipkin_annotations ADD INDEX(`a_type`) COMMENT 'for getTraces'; ALTERTABLE zipkin_annotations ADD INDEX(`a_key`) COMMENT 'for getTraces'; ALTERTABLE zipkin_annotations ADD INDEX(`trace_id`, `span_id`, `a_key`) COMMENT 'fordependencies job'; CREATETABLE IF NOT EXISTS zipkin_dependencies ( `day` DATE NOT NULL, `parent` VARCHAR(255) NOT NULL, `child` VARCHAR(255) NOT NULL, `call_count` BIGINT )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci; ALTERTABLE zipkin_dependencies ADD UNIQUE KEY(`day`, `parent`, `child`);
建立成功後,查詢結果。
mysql> show tables; +---------------------+ | Tables_in_mytestdb | +---------------------+ | zipkin_annotations | | zipkin_dependencies | | zipkin_spans | +---------------------+ 3 rows in set (0.00 sec)
啟動zipkin-dependencies
最開始我的密碼是【四個字母一個感嘆號一個數字】,再執行啟動命令時,密碼那塊給我報錯自動換成【四個字母rm -f】 ,我修改成【四個字母一個#號一個數字】就能執行了
執行成功後,依然提示Access denied for user 'root'@'localhost' (using password: NO) ,但我在linux的命令中直接用mysql -u root -p相同密碼是可以登入成功的。所以問題出現在哪呢?
[root@izwz9fwifc2eniq3lbdzmgz cusD]# STORAGE_TYPE=mysql MYSQL_HOST=localhost MYSQL_TCP_PORT=3306 MYSQL_DB=mytestdb MYSQL_USER=XXXXX MYSQL_PASS=XXXXX java -Xmx1024m -Xms1024m -jar zipkin-dependencies.jar Exception in thread "main" java.lang.RuntimeException: java.sql.SQLInvalidAuthorizationSpecException: Access denied for user 'root'@'localhost' (using password: NO) at zipkin2.dependencies.mysql.MySQLDependenciesJob.hasTraceIdHigh(MySQLDependenciesJob.java:233) at zipkin2.dependencies.mysql.MySQLDependenciesJob.run(MySQLDependenciesJob.java:184) at zipkin2.dependencies.ZipkinDependenciesJob.main(ZipkinDependenciesJob.java:65) Caused by: java.sql.SQLInvalidAuthorizationSpecException: Access denied for user 'root'@'localhost' (using password: NO) at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:173) at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:110) at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1115) at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:502) at org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:154) at org.mariadb.jdbc.Driver.connect(Driver.java:86) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at zipkin2.dependencies.mysql.MySQLDependenciesJob.hasTraceIdHigh(MySQLDependenciesJob.java:229) ... 2 more Caused by: java.sql.SQLException: Access denied for user 'root'@'localhost' (using password: NO) Current charset is UTF-8. If password has been set using other charset, consider using option 'passwordCharacterEncoding' at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.authentication(AbstractConnectProtocol.java:862) at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.handleConnectionPhases(AbstractConnectProtocol.java:785) at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:456) at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1111) ... 8 more
追溯原始碼
逼不得已,走上檢視原始碼之路,Idea開啟zipkin-dependencies/mysql原始碼,檢視相關部分程式碼。
public static final class Builder { Map<String, String> sparkProperties = ImmutableMap.of( "spark.ui.enabled", "false" ); String db = getEnv("MYSQL_DB", "zipkin"); String host = getEnv("MYSQL_HOST", "localhost"); int port = Integer.parseInt(getEnv("MYSQL_TCP_PORT", "3306")); String user = getEnv("MYSQL_USER", ""); String password = getEnv("MYSQL_PASS", ""); int maxConnections = Integer.parseInt(getEnv("MYSQL_MAX_CONNECTIONS", "10")); boolean useSsl = Boolean.parseBoolean(getEnv("MYSQL_USE_SSL", "false")); // local[*] master lets us run & test the job locally without setting a Spark cluster String sparkMaster = getEnv("SPARK_MASTER", "local[*]"); // By default the job only works on traces whose first timestamp is today long day = midnightUTC(System.currentTimeMillis()); /** *為減少篇幅,中間設定屬性部分程式碼省略。 */ public MySQLDependenciesJob build() { return new MySQLDependenciesJob(this); } } /** *為減少篇幅,中間部分程式碼省略。 */ MySQLDependenciesJob(Builder builder) { this.db = builder.db; this.day = builder.day; SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd"); df.setTimeZone(TimeZone.getTimeZone("UTC")); this.dateStamp = df.format(new Date(builder.day)); this.url = new StringBuilder("jdbc:mysql://") .append(builder.host).append(":").append(builder.port) .append("/").append(builder.db) .append("?autoReconnect=true") .append("&useSSL=").append(builder.useSsl).toString(); this.user = builder.user; this.password = builder.password; this.conf = new SparkConf(true) .setMaster(builder.sparkMaster) .setAppName(getClass().getName()); if (builder.jars != null) conf.setJars(builder.jars); for (Map.Entry<String, String> entry : builder.sparkProperties.entrySet()) { conf.set(entry.getKey(), entry.getValue()); } this.logInitializer = builder.logInitializer; } void saveToMySQL(List<DependencyLink> links) { try (Connection con = DriverManager.getConnection(url, user, password)) { PreparedStatement replace = con.prepareStatement( "REPLACE INTO zipkin_dependencies (day, parent, child, call_count, error_count) VALUES (?,?,?,?,?)"); for (DependencyLink link : links) { replace.setDate(1, new java.sql.Date(day)); replace.setString(2, link.parent()); replace.setString(3, link.child()); replace.setLong(4, link.callCount()); replace.setLong(5, link.errorCount()); replace.executeUpdate(); } } catch (SQLException e) { throw new RuntimeException("Could not save links " + links, e); } }
然並卵,看完之後,沒看出明顯問題。難道還是我自己的mysql配置問題?還是啟動部分的引數問題?程式碼部分也是有些疑惑,password和root為什麼沒放進url裡,難道是為了安全考慮麼?
this.url = new StringBuilder("jdbc:mysql://") .append(builder.host).append(":").append(builder.port) .append("/").append(builder.db) .append("?autoReconnect=true") .append("&useSSL=").append(builder.useSsl).toString(); this.user = builder.user; this.password = builder.password;
文中還提到 Current charset is UTF-8. If password has been set using other charset, consider using option 'passwordCharacterEncoding',編碼格式是否有不同呢?
檢視mysql資料庫及表編碼格式mysql> show variables like 'character_set_database'; +------------------------+---------+ | Variable_name | Value | +------------------------+---------+ | character_set_database | utf8mb4 | +------------------------+---------+ 1 row in set (0.01 sec)
參考連結
總結
由於啟動zipkin-dependencies連結mysql報Access denied for user 'root'@'localhost' (using password: NO) 錯誤,本次持久化之路最終失敗。但由於我直接使用【mysql -u 使用者 -p】是能登入成功的,所以我猜測了以下原因:
- 客戶端自己的bug,和我伺服器mysql版本不相容?
- 編碼問題,編碼兩者不符?
- 使用者名稱和密碼沒有共享全域性,只對一個數據庫有效?
別人的博文是面向教學成功程式設計,我的是面向失敗程式設計,也別有一番趣味。留下疑問,待日後解決調。雖然失敗了,但我又收集了一堆連結,增添了mysql一些故障解決的認識。
今天換了一款個人很喜歡的面板,會根據h1,h2自動生成目錄,之前的博文我也都檢查了下,有很大失位的我都調整了過來,不標準的暫時不改了,我以後的博文都按照要求的格式寫,排版美觀度提升了很多,感謝作者bndong,如果有打算使用這個面板的,一定要開啟控制元件顯示公告。