【采坑小计】prometheus的remote write协议遇到的问题

没有读懂源码以前,无脑试错总是效率很低的!

1.thanos receiver报store locally for endpoint : conflict

接口返回的日志:

store locally for endpoint : conflict
code=409

thanos receiver上的debug日志:

2021-08-20_20:49:43 level=debug ts=2021-08-20T12:49:43.941114321Z caller=handler.go:351 component=receive component=receive-handler msg="failed to handle request" err="store locally for endpoint : conflict"
2021-08-20_20:57:23 level=debug ts=2021-08-20T12:57:23.414813105Z caller=writer.go:91 component=receive component=receive-writer msg="Duplicate sample for timestamp" lset="{__name__="monitorid_73569379", instance="send_client", job="xxxx"}" sample="unsupported value type"
2021-08-20_20:57:23 level=warn ts=2021-08-20T12:57:23.414856366Z caller=writer.go:135 component=receive component=receive-writer msg="Error on ingesting samples with different value but same timestamp" numDropped=1

原因是:短时间内,同样的时间戳,同样的监控项,上报了两次

2. 不填写sample中的timestamp

thanos receiver上的debug日志:

2021-08-20_20:44:49 level=debug ts=2021-08-20T12:44:49.190297867Z caller=handler.go:351 component=receive component=receive-handler msg="failed to handle request" err="store locally for endpoint : conflict"
2021-08-20_20:44:49 level=debug ts=2021-08-20T12:44:49.194313101Z caller=writer.go:94 component=receive component=receive-writer msg="Out of bounds metric" lset="{__name__="monitorid_73599067", instance="send_client", job="trpc"}" sample="unsupported value type"
2021-08-20_20:44:49 level=warn ts=2021-08-20T12:44:49.194335493Z caller=writer.go:139 component=receive component=receive-writer msg="Error on ingesting samples that are too old or are too far into the future" numDropped=1
原文地址:https://www.cnblogs.com/ahfuzhang/p/15168173.html