savsgio/kratgo 基于fasthttp 开发的一个cache proxy 服务

savsgio/kratgo 基于fasthttp 开发的web cache 服务,包含以下特性

支持的特性

  • cache proxy
  • lb (顺序,然后轮转模式,没有权重)
  • 配置cache 规则
  • 配置http 请求头
  • 管理api 支持,基于管理api 我们可以进行cache 清理

参考使用

  • docker-compose 文件
version: "3"
services:
  proxy: 
    image: savsgio/kratgo
    command: -config /etc/config.yaml
    ports:
    - "6081:6081"
    volumes: 
     - "./config.yaml:/etc/config.yaml"
  • 配置
###########################
#   KRATGO CONFIGURATION
###########################
# IMPORTANT: Be careful with the tabulation indentation
# --- Variables ---
# $(method) : request method
# $(host) : request host
# $(path) : request path
# $(contentType) : response backend's content type
# $(statusCode) : response backend's status code
# $(req.header::<NAME>) : request header name
# $(resp.header::<NAME>) : response header name
# $(cookie::<NAME>) : request cookie name
# --- Operators ---
# Modifiers: + - / * & | ^ ** % >> <<
# Comparators: > >= < <= == != =~ !~
# Logical ops: || &&
# Numeric constants, as 64-bit floating point (12345.678)
# String constants (single quotes: 'foobar')
# Date constants (single quotes, using any permutation of RFC3339, ISO8601, ruby date, or unix date; date parsing is automatically tried with any string constant)
# Boolean constants: true false
# Parenthesis to control order of evaluation ( )
# Arrays (anything separated by , within parenthesis: (1, 2, 'foo'))
# Prefixes: ! - ~
# Ternary conditional: ? :
# Null coalescence: ??
# --- Log ---
# Log level: fatal | error | warning | info | debug
# Log output:
#   - console: Write output in standard error
#   - <file path>: Write output in log file
logLevel: debug
logOutput: ./kratgo.log
# --- Cache ---
# ttl: Cache expiration in minutes
# cleanFrequency: Interval in minutes between removing expired entries (clean up)
# maxEntries: Max number of entries in cache. Used only to calculate initial size for cache
# maxEntrySize: Max size of entry in bytes
# hardMaxCacheSize: Limit for cache size in MB (Default value is 0 which means unlimited size)
cache:
  ttl: 10
  cleanFrequency: 1
  maxEntries: 600000
  maxEntrySize: 500
  hardMaxCacheSize: 0
# --- Invalidator ---
# maxWorkers: Maximum workers to execute invalidations
invalidator:
  maxWorkers: 5
# --- Proxy ---
# addr: IP and Port of Kratgo
# backendAddrs: Array with "addr:port" of the backends
# response: Configuration to manipulate reponse (Optional)
#   headers:
#     set: Configuration to SET headers from response (Optional)
#       - name: Header name
#         value: Value of header
#         if: Condition to set this header (Optional)
#
#     unset: Configuration to UNSET headers from response (Optional)
#       - name: Header name
#         if: Condition to unset this header (Optional)
#
# nocache: Conditions to not save in cache the backend response (Optional)
proxy:
  addr: 0.0.0.0:6081
  backendAddrs:
    [
       <address>
    ]
  response:
    headers:
      set:
        - name: X-Kratgo
          value: true
      unset:
        - name: Set-Cookie
          if: $(req.header::X-Requested-With) != 'XMLHttpRequest'
  nocache:
    - $(req.header::X-Requested-With) == 'XMLHttpRequest'
# --- Admin ---
# addr: IP and Port of admin api
admin:
  addr: 0.0.0.0:6082
  • 效果

同时,如果断开网络链接,你会发现网站依然还能访问(因为cache了)

基于fasthttp 开发一个自己proxy 的简单代码

 
package main
import (
  "flag"
  "fmt"
  "log"
  "github.com/valyala/fasthttp"
)
var (
  add = flag.String("add", "localhost", "proxy address")
)
type myLog struct {
  add string
}
func (l myLog) Printf(format string, args ...interface{}) {
  log.Printf(format, args...)
}
func (l myLog) myhandler(ctx *fasthttp.RequestCtx) {
  backend := fasthttp.HostClient{
    Addr: l.add,
  }
  backend.Do(&ctx.Request, &ctx.Response)
}
func main() {
  flag.Parse()
  fmt.Println(*add)
  log := myLog{
    add: *add,
  }
  s := fasthttp.Server{
    Handler:      log.myhandler,
    LogAllErrors: true,
    Logger:       log,
  }
  s.ListenAndServe(":8000")
}

源码简单说明

savsgio/kratgo 的源码并不难,对于cache proxy 基于fasthttp 的Server提供,对于cache 部分基于bigcache
Server handler 使用了fasthttp 的fasthttp.HostClient,配置解析部分基于yaml,指定语法解析(配置自定义参数)
使用Knetic/govaluate,同时为了性能作者很多地方使用了sync.Pool ,lb 部分实现很简单,就是数据索引的移动
当超过数据最大值得时候,重置索引为0,同时对于cache 管理,作者基于channel 实现了一个校验任务,实际可以
看看源码,实际上以后也分享过gregjones/httpcache的开源实现,也是一个不错的选择

参考资料

https://github.com/savsgio/kratgo
https://github.com/Knetic/govaluate
https://github.com/valyala/fasthttp
https://github.com/allegro/bigcache
https://github.com/gregjones/httpcache

原文地址:https://www.cnblogs.com/rongfengliang/p/14082424.html