install client pip install shadowsocks client.json
{ "server":"server-ip", "server_port":8000, "local_port":3050, "password":"your-password", "timeout":600, "method":"aes-256-cfb" } { "server":"your_server_ip", #ss服务器IP "server_port":your_server_port, #端口 "local_address": "127.0.0.1", #本地ip "local_port":1080, #本地端口 "password":"your_server_passwd",#连接ss密码 "timeout":300, #等待超时 "method":"rc4-md5", #加密方式 "fast_open": false, # true 或 false。如果你的服务器 Linux 内核在3.7+，可以开启 fast_open 以降低延迟。开启方法： echo 3 > /proc/sys/net/ipv4/tcp_fastopen 开启之后，将 fast_open 的配置设置为 true 即可 "workers": 1 # 工作线程数 } sudo apt-get install privoxy /etc/privoxy/config
listen-address 127.0.0.1:8118 forward-socks5 / 127.0.0.1:1080 . systemctl restart privoxy.
特性 与hadoop集成，支持MR数据读取 二级索引 支持长行（最多20亿的列） 动态schema更改 bulk load 其他数据源如hadoop with sstableloader， CSV importing with cqlsh DTCS优化时序数据性能 DB CATEGORY BY CAP CA To primarily support consistency and availability means that you’re likely using two-phase commit for distributed transactions. It means that the system will block when a network partition occurs, so it may be that your system is limited to a single data center cluster in an attempt to mitigate this.
为什么要使用OIDC 只使用基本OAUTH2认证是不安全因为无法保证与用户的访问令牌是真实的。使用 OIDC可以做到这一点
https://security.stackexchange.com/questions/37818/why-use-openid-connect-instead-of-plain-oauth2/260519#260519
hasRole a role is just an authority with a special ROLE_ prefix. So in Spring security 3 @PreAuthorize(“hasRole(‘ROLE_XYZ’)“) is the same as @PreAuthorize(“hasAuthority(‘ROLE_XYZ’)“) and in Spring security 4 @PreAuthorize(“hasRole(‘XYZ’)”) is the same as @PreAuthorize(“hasAuthority(‘ROLE_XYZ’)“).
http://localhost:8080/oauth/token
curl -u eagleeye:thisissecret -i -H ‘Accept:application/json’ -d “grant_type=password&scope=webclient&username=will&password=pass” -H “Content-Type: application/x-www-form-urlencoded” -X POST http://localhost:8080/oauth/token
access protected resource
push to a mirror repository push to github at same time when a commit is pushed to gitlab
Protected Branches By default, protected branches are designed to:
prevent their creation, if not already created, from everybody except Maintainers prevent pushes from everybody except Maintainers prevent anyone from force pushing to the branch prevent anyone from deleting the branch Project members permissions NOTE:
In GitLab 11.0, the Master role was renamed to Maintainer The following table depicts the various user permission levels in a project.
rabbit kafka 创建时间 2007 2011 开发语言 erlang scala AMQP SUPPORT NO AGENT SMART(broker-centric) keeps track of consumer state dumb(producer-centric) 存储空间 in-memory disk INGRESS VOLUME 20K messages/sec 100k/sec messages/sec CONSUMERS mostly online(balancing load to many consumer) online and batch consumer ROUTING exchange, binding simple history N/A replay(删除by size 或时间) 数据压缩 N Y SPRING SUPPORT weak strong 安全 RBAC backed by a built-in data store, LDAP JAAS role based access 管理 Web 和 CLI JMX 和 CLI
mysqladmin status MySQL is multithreaded, so there may be many clients issuing queries for a given table simultaneously. To minimize the problem with multiple client sessions having different states on the same table, the table is opened independently by each concurrent session. This uses additional memory but normally increases performance
The table_open_cache and max_connections system variables affect the maximum number of files the server keeps open. If you increase one or both of these values, you may run up against a limit imposed by your operating system on the per-process number of open file descriptors.
innodb-memcached-multiple-get-range-query native partitioning in-place APIs
[mysqld] server-id = 2 relay-log-index = slave-relay-bin.index relay-log = slave-relay-bin mysql>CHANGE MASTER TO MASTER_HOST = 'db2',MASTER_PORT = 3306, MASTER_USER = 'repl_user', MASTER_PASSWORD = 'xyzzy'; Connecting the Master mysql> START SLAVE;
Server and Operating System Kernel – vm.swappiness Disables swapping completely while 1 causes the kernel to perform the minimum amount of swapping
# Set the swappiness value as root echo 1 > /proc/sys/vm/swappiness # Alternatively, using sysctl sysctl -w vm.swappiness=1 # Verify the change cat /proc/sys/vm/swappiness 1 # Alternatively, using sysctl sysctl vm.swappiness vm.swappiness = 1 Filesystems – XFS/ext4/ZFS FILE SIZE mount option EXT4 16TB noatime,data=writeback,barrier=0,nobh,errors=remount-ro XFS 8EiB defaults,nobarrier Disk Subsystem – I/O scheduler Most modern Linux distributions come with noop or deadline I/O schedulers by default, both providing better performance than the cfq and anticipatory ones
mysql 8 测试环境快速搭建(WSL/root远程访问) sudo apt install -y mysql-server mysql --version sudo mysql mysql>CREATE USER 'root'@'%' IDENTIFIED BY '123'; mysql>GRANT ALL PRIVILEGES ON *.* TO 'root'@'%'; mysql>flush PRIVILEGES mysql.conf.d/mysqld.cnf :32:bind-address = 0.0.0.0
sudo service mysql restart Create the root user (yes, a new user because what exists is ‘root@localhost’ which is local access only)
root用户本地登录 本地登录使用系统认证(auth_socket)
sudo mysql root用户远程登录 mysql -u root -p'123' -h 192.168.168.128 mysql> select user,host,plugin from mysql.
mysql install ubuntu 16.04 install mysql 5.7 at default
sudo apt-get update sudo apt-get install mysql-server Enable root remote connection
mysql -u root -p mysql>GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '<password>' WITH GRANT OPTION; mysql>FLUSH PRIVILEGES; SHOW current setting mysql> SHOW VARIABLES WHERE Variable_name LIKE 'innodb%'; MySQL 5.7 has significantly better default values. the following variables are set by default:
[mysqld] innodb_buffer_pool_instances=8 innodb_flush_method=O_DIRECT setting of mysql 5.
whether MySQL Server supports partitioning
mysql -u root -p123456 -e "SHOW PLUGINS;" |grep partition
show the last queries mysql>SET GLOBAL log_output = 'FILE'; mysql>SET GLOBAL general_log = 'ON'; mysql>SHOW VARIABLES WHERE Variable_name LIKE 'general_log_file' Disable ONLY_FULL_GROUP_BY SHOW VARIABLES WHERE Variable_name LIKE 'sql_mode'; +---------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Variable_name | Value | +---------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | sql_mode | ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION | [mysqld] sql_mode = "STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" mysql> SHOW CREATE TABLE <tablename>;
[mysqld] log-bin = master-bin log-bin-index = master-bin.index server-id = 1 Grant the user to retrieve the binary log from the master
mysql>CREATE USER repl_user; GRANT REPLICATION SLAVE ON *.* TO repl_user IDENTIFIED BY 'xyzzy';
key_buffer_size the size of the index buffers held in memory, which affects the speed of index reads
recommend: 25% or more of the available server memory
A good way to determine whether to adjust the value is to compare the key_read_requests value, which is the total value of requests to read an index, and the key_reads values, the total number of requests that had to be read from disk.
push docker images to ali registry-mirrors https://cr.console.aliyun.com
#!/usr/bin/env bash docker login --username=wubigo registry.cn-beijing.aliyuncs.com docker images | grep v1.13 | awk '{ print $1 }' | sed --expression=s'/K8S.gcr.io\///' | xargs -i -t docker tag K8S.gcr.io/{}:v1.13.3 registry.cn-beijing.aliyuncs.com/co1/{}:v1.13.3 docker images |grep "registry.cn-beijing.aliyuncs.com"| awk '{ print $1 }'| sed --expression=s'/registry.cn-beijing.aliyuncs.com\/co1\///' | xargs -i -t docker push registry.cn-beijing.aliyuncs.com/co1/{}:v1.13.3 docker push through cache #!/usr/bin/env bash if [ -z "$VM" ]; then VM = t1 echo "VAR VM is not set" exit fi tee daemon.
LINUX shell常用工具提供强大的功能，在日常中熟练掌握能给我 带来不少动能
grep cat find head/tail wc awk shuf 查找 在logs目录下查找所有包含2010_05_02的日志文件
ls logs/ | grep 2010_05_02 pip freeze | grep scipy scipy==1.1.0 grep -oP "'[\w]+ == [\d.]+'" setup.py scipy == 1.1.0 #
find . -name '..*swp' -delete awk head -n 1 data.csv | awk -F ',' '{print NF}' shuf 从数据集中随机取50个样本
cat big_csv.csv | shuf | head -n 50 > sample_csv.csv