杂记

Typora 上传图片到博客园

import requests
import json
from sys import argv

uploadUrl = 'https://upload.cnblogs.com/imageuploader/processupload?host=www.cnblogs.com'

headers = {
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36",
    "cookie": "__gads=ID=0df8844e3171cf5d:T=1626703371:S=ALNI_MbCQzZlPNaCt56Y3ZVBMgBgxo-LiA; UM_distinctid=17b02249c8337a-021de176cab159-3f3a5e08-1fa400-17b02249c84437; _ga_3Q0DVSGN10=GS1.1.1629294955.2.1.1629294963.0; _ga=GA1.2.1874840476.1626703371; .Cnblogs.AspNetCore.Cookies=CfDJ8NACB8VE9qlHm6Ujjqxvg5A9JzQnWwTQwllXBS3jsuzbCTLy2gQscie0pLfzrbMgn704PWOBvn29tPVe9SrWwTZDRV8hTTISIxYAGXMLFead8T1oy-hTGkXxDwlskdW8yHcTMpvI481DuI0BGX4LeP5jib45Xj6vGjzibGkPiBou2eNyGQt3tVlMbHaSKcAspkeQmALgMVJEB-9ZoiijWgHo8M3_6F7RIUNgXml1yjrhu9bvWHeD6-tRHOigQRAQjS4QBG0hP_LXMT7RcXKFdy8AE2ZuCqTNZzrrNjwqXM7TpNw7uXpo2t0lo7tJ68MwJsoQPDkV3XIBIqKXyhc7bbzkm-OxcdlBNseZaxVm0gge1oRIIOo8NZoWePmQjjfF2TmwHgpdoZfH6kkc1FLCsB8THIUuAQGee9p7iK3Gx1CEmAfhILRECIwPxE0wxPdjl6FWvIANUJ8rELfJfUWKfxhVhhU3So1Ywlgv9tPTKegzgUSooWyzn7Fnlb4cLkGrXM4ncoAHUOFzwpslnrwS4vp3-12wXeOb2ggb7NfVL3TlEPojFPPj1hHDRTkeOfGo9A; _gid=GA1.2.205401666.1631025338; affinity=1631273060.452.42.609693" 
}

# 类型映射
mimeMapping = {".png": 'image/png', '.gif': 'image/gif', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg'}

for i in argv[1:]:
    # 图片地址参数
    imgPath = i

    # 对应的mime
    mime = imgPath[imgPath.rindex("."):]

    file = [
        ("",("fileName", open(imgPath, "rb"), mimeMapping[mime]))
    ]

    response = requests.post(uploadUrl,headers = headers,files = file)

    data = json.loads(response.text)
    print(data['message'])

偏好设置中需要配置为上传服务

Flink on Yarn 集群报错

报错内容

 Exception in thread "Thread-5" java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.
        at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.ensureInner(FlinkUserCodeClassLoaders.java:164)
        at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.getResource(FlinkUserCodeClassLoaders.java:183)
        at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2737)
        at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:2993)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2952)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2925)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2805)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1199)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1787)
        at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
        at org.apache.hadoop.util.ShutdownHookManager.shutdownExecutor(ShutdownHookManager.java:145)
        at org.apache.hadoop.util.ShutdownHookManager.access$300(ShutdownHookManager.java:65)
        at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:102)

解决方案

在conf/flink-conf.yaml,大约在192行添加

classloader.check-leaked-classloader: false

重新执行 Flink per-job cluster 调用WordCount样例

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 examples/batch/WordCount.jar

正常打印输出结果.

Flink On Session 模式

先创建一个on Yarn的Flink 容器

bin/yarn-session.sh -jm 1024m -tm 1024m -d

执行测试PI脚本

bin/flink run ./examples/batch/WordCount.jar

查看默认配置信息

more /tmp/.yarn-properties-root

杀掉yarn上的Flink 容器

yarn application -kill application_1631228277006_0008