Fabric 错误异常处理

Fabric是Python编写的自动化运维工具,使用fabric批量去机器上面执行shell脚本的时候,可能会遇到机器故障等情况,造成执行中断,那么能否忽略或者说爆出异常之后继续执行呢?那就需要fabric的异常处理,具体实现参见下面

import logging

try:
    1/0
except Exception as e:
    logging.exception(e)

错误异常打印如下:

ERROR:root:division by zero
Traceback (most recent call last):
  File "/home/padraic/Dropbox/python/py3/size.py", line 105, in <module>
    1/0
ZeroDivisionError: division by zero

Process finished with exit code 0

《一分钟经理人》读书笔记

一分钟经理人这本书比较薄,上下班通勤路上就看完了,包含内容归纳的也比较简洁,印象比较深的是其中一页插图,分别对应着开始时要考虑一分钟目标、成功时要考虑一分钟表扬和失败时应考虑一分钟更正,简洁但不简单,又看了一下Kindle上阅读时的标注,回顾一下书上的内容。
Read more

dependencies.dependency.version missing问题解决

Spring Boot打算引入Redis但是导入依赖包之后却报如下错误

        <dependency>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-redis</artifactId>
        </dependency>
Project build error: 'dependencies.dependency.version' for org.springframework.boot:spring-boot-starter-redis:jar is missing.

同时pom配置文件中位置,报如下位置:

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.9.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
Multiple annotations found at this line:
    - For artifact {org.springframework.boot:spring-boot-starter-redis:null:jar}: The version cannot be empty. (org.apache.maven.plugins:maven-resources-plugin:2.6:testResources:default-
     testResources:process-test-resources) org.apache.maven.artifact.InvalidArtifactRTException: For artifact {org.springframework.boot:spring-boot-starter-redis:null:jar}: The version cannot be empty. at 
     org.apache.maven.artifact.DefaultArtifact.validateIdentity(DefaultArtifact.java:148) at org.apache.maven.artifact.DefaultArtifact.<init>(DefaultArtifact.java:123) at 
     org.apache.maven.artifact.factory.DefaultArtifactFactory.createArtifact(DefaultArtifactFactory.java:157) at 
     org.apache.maven.artifact.factory.DefaultArtifactFactory.createDependencyArtifact(DefaultArtifactFactory.java:57) at 
     org.apache.maven.project.artifact.MavenMetadataSource.createDependencyArtifact(MavenMetadataSource.java:328) at 
     org.apache.maven.project.artifact.MavenMetadataSource.createArtifacts(MavenMetadataSource.java:503) at 
     org.apache.maven.lifecycle.internal.DefaultProjectArtifactFactory.createArtifacts(DefaultProjectArtifactFactory.java:50) at 
     org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:119) at 
     org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:257) at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:330) at 
     org.eclipse.m2e.core.internal.embedder.MavenImpl11.call(MavenImpl.java:1362) at org.eclipse.m2e.core.internal.embedder.MavenImpl11.call(MavenImpl.java:1) at 
     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176) at 
     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112) at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1360) at 
     org.eclipse.m2e.core.project.configurator.MojoExecutionBuildParticipant.build(MojoExecutionBuildParticipant.java:52) at 
     org.eclipse.m2e.core.internal.builder.MavenBuilderImpl.build(MavenBuilderImpl.java:137) at org.eclipse.m2e.core.internal.builder.MavenBuilder1.method(MavenBuilder.java:172) at     org.eclipse.m2e.core.internal.builder.MavenBuilder1.method(MavenBuilder.java:1) at org.eclipse.m2e.core.internal.builder.MavenBuilderBuildMethod11.call(MavenBuilder.java:115) at     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176) at     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112) at org.eclipse.m2e.core.internal.builder.MavenBuilderBuildMethod1.call(MavenBuilder.java:
     105) at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176) at     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151) at     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99) at org.eclipse.m2e.core.internal.builder.MavenBuilderBuildMethod.execute(MavenBuilder.java:
     86) at org.eclipse.m2e.core.internal.builder.MavenBuilder.build(MavenBuilder.java:200) at org.eclipse.core.internal.events.BuildManager2.run(BuildManager.java:734) at     org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:205) at     org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:245) at org.eclipse.core.internal.events.BuildManager1.run(BuildManager.java:300) at 
     org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:303) at 
     org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:359) at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:382) at 
     org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:144) at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:235) at 
     org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
    - For artifact {org.springframework.boot:spring-boot-starter-redis:null:jar}: The version cannot be empty. (org.apache.maven.plugins:maven-resources-plugin:2.6:resources:default-
     resources:process-resources) org.apache.maven.artifact.InvalidArtifactRTException: For artifact {org.springframework.boot:spring-boot-starter-redis:null:jar}: The version cannot be empty. at 
     org.apache.maven.artifact.DefaultArtifact.validateIdentity(DefaultArtifact.java:148) at org.apache.maven.artifact.DefaultArtifact.<init>(DefaultArtifact.java:123) at 
     org.apache.maven.artifact.factory.DefaultArtifactFactory.createArtifact(DefaultArtifactFactory.java:157) at 
     org.apache.maven.artifact.factory.DefaultArtifactFactory.createDependencyArtifact(DefaultArtifactFactory.java:57) at 
     org.apache.maven.project.artifact.MavenMetadataSource.createDependencyArtifact(MavenMetadataSource.java:328) at 
     org.apache.maven.project.artifact.MavenMetadataSource.createArtifacts(MavenMetadataSource.java:503) at 
     org.apache.maven.lifecycle.internal.DefaultProjectArtifactFactory.createArtifacts(DefaultProjectArtifactFactory.java:50) at 
     org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:119) at 
     org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:257) at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:330) at 
     org.eclipse.m2e.core.internal.embedder.MavenImpl11.call(MavenImpl.java:1362) at org.eclipse.m2e.core.internal.embedder.MavenImpl11.call(MavenImpl.java:1) at 
     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176) at 
     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112) at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1360) at 
     org.eclipse.m2e.core.project.configurator.MojoExecutionBuildParticipant.build(MojoExecutionBuildParticipant.java:52) at 
     org.eclipse.m2e.core.internal.builder.MavenBuilderImpl.build(MavenBuilderImpl.java:137) at org.eclipse.m2e.core.internal.builder.MavenBuilder1.method(MavenBuilder.java:172) at     org.eclipse.m2e.core.internal.builder.MavenBuilder1.method(MavenBuilder.java:1) at org.eclipse.m2e.core.internal.builder.MavenBuilderBuildMethod11.call(MavenBuilder.java:115) at     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176) at     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112) at org.eclipse.m2e.core.internal.builder.MavenBuilderBuildMethod1.call(MavenBuilder.java:
     105) at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176) at     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151) at     org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99) at org.eclipse.m2e.core.internal.builder.MavenBuilderBuildMethod.execute(MavenBuilder.java:
     86) at org.eclipse.m2e.core.internal.builder.MavenBuilder.build(MavenBuilder.java:200) at org.eclipse.core.internal.events.BuildManager2.run(BuildManager.java:734) at     org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:205) at     org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:245) at org.eclipse.core.internal.events.BuildManager1.run(BuildManager.java:300) at 
     org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:303) at 
     org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:359) at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:382) at 
     org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:144) at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:235) at 
     org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

解决方案

排查一:

是否引入了<parent>标签,如果未引入请引入,或者主动加上version

排查二:

是否dependences包含在了dependencyManagement标签内

排查三:

其中网上搜索了一下很多答案集中在是否引入了parent,但是parent是完整引入的,最终发现原来是因为使用的Spring Boot 1.5,而Spring Boot 1.2之后不再支持spring-boot-starter-redis,更换之后就OK了,至于这两个有何不一致,没什么不一致,为何名称变了,大概可能是为了统一,记录一下,如果遇到的小伙伴麻烦检查一下,是否将 spring-boot-starter-data-redis 写成了spring-boot-starter-redis 。

解决Error: pg_config executable not found问题

最近安装sentry时,需要安装postgresql,但是安装过程中报如下问题

Error: pg_config executable not found.

最终查看pg_config是存在的

sudo find / -name "pg_config" -print

但是path里面没有,直接导入pa_home中bin目录的所有命令解决

export PG_HOME=/usr/pgsql-9.6
export PATH=PATH:PG_HOME/bin

解决SOAP客户端在请求https时设置超时时间无效导致进程卡死问题

目前有一个进程服务脚本是不断查询渠道的接口,但是历史问题是有时订单量大的时候进程会卡死,这次遇到了进行排查一下:

首先获取该进程ID

ps -aux | grep QueryABC.php

sync360  11115  0.0  0.0   6564   864 ?        Ss   14:00   0:00 /bin/sh -c /usr/local/bin/php /xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/QueryABC.php BILL99DF 10-8>> /home
sync360  11124  0.0  0.4 361628 17296 ?        S    14:00   0:04 /usr/local/bin/php /xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/QueryABC.php BILL99DF 10-8
sync360  25230  0.0  0.0  63384   872 pts/0    S+   15:28   0:00 grep QueryABC.php

strace查看该进程正在持续的状态

sudo strace -T -tt -e trace=all -p 11124
[sudo] password for ancongcong: 
Process 11124 attached - interrupt to quit
15:33:07.259044 read(9, 

lsof查看进程的所使用的文件

lsof -p 11124

....
php     11124 sync360  mem    REG        8,1    23736    3211320 /lib64/libnss_dns-2.5.so
php     11124 sync360    0r  FIFO        0,6          1522728709 pipe
php     11124 sync360    1w   REG        8,1  4088819    1869737 /xxxx/xxxx/xxxx/xxxx/logs/QueryABC.log
php     11124 sync360    2w  FIFO        0,6          1522728710 pipe
php     11124 sync360    3w   CHR        1,3                 982 /dev/null
php     11124 sync360    4u  IPv4 1522728838                 TCP 211.151.122.234:46004->10.117.128.47:rtmp-port (CLOSE_WAIT)
php     11124 sync360    5wW  REG        8,1        0    2704363 /xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/xxxx/lockfile/QueryABC.php.BILL99DF.10-8
php     11124 sync360    6u  IPv4 1522728841                 TCP 211.151.122.234:51019->10.117.128.46:rtmp-port (CLOSE_WAIT)
php     11124 sync360    7w   REG        8,1 31960384    1869789 /xxxx/xxxx/xxxx/xxxx/logs/XXXX_info.log.20180118
php     11124 sync360    8w   REG        8,1 18151722    1869806 /xxxx/xxxx/xxxx/xxxx/logs/XXXX_QRY_info.log.20180118
php     11124 sync360    9u  IPv4 1522729884                 TCP 211.151.122.234:54976->61.152.114.130:https (ESTABLISHED)
sudo netstat -tunpa | grep 11124
tcp        0      0 211.151.122.234:54976       61.152.114.130:443          ESTABLISHED 11124/php           
tcp        1      0 211.151.122.234:51019       10.117.128.46:3500          CLOSE_WAIT  11124/php           
tcp        1      0 211.151.122.234:46004       10.117.128.47:3500          CLOSE_WAIT  11124/php      

可以发现最终是停留在https的链接建立,等待获取数据,查看此处代码

ini_set('default_socket_timeout',30);
scOptions = array('connection_timeout' => 30);clientObj = new SoapClient( wsdl ,scOptions);

当前版本php较老,这里是有个bug的在https链接请求时SOAPClient的超时时间是不生效,最终采取如下方案解决此问题:

复写SOAPClient,在https时候使用curl来完成请求解决问题

<?php
class SoapClientTimeout extends SoapClient
{
    private timeout;

    public function __setTimeout(timeout)
    {
        if (!is_int(timeout) && !is_null(timeout))
        {
            throw new Exception("Invalid timeout value");
        }

        this->timeout =timeout;
    }

    public function __doRequest(request,location, action,version, one_way = FALSE)
    {
        if (!this->timeout)
        {
            // Call via parent because we require no timeout
            response = parent::__doRequest(request, location,action, version,one_way);
        }
        else
        {
            // Call via Curl and use the timeout
            curl = curl_init(location);

            curl_setopt(curl, CURLOPT_VERBOSE, FALSE);
            curl_setopt(curl, CURLOPT_RETURNTRANSFER, TRUE);
            curl_setopt(curl, CURLOPT_POST, TRUE);
            curl_setopt(curl, CURLOPT_POSTFIELDS, request);
            curl_setopt(curl, CURLOPT_HEADER, FALSE);
            curl_setopt(curl, CURLOPT_HTTPHEADER, array("Content-Type: text/xml"));
            curl_setopt(curl, CURLOPT_TIMEOUT, this->timeout);response = curl_exec(curl);

            if (curl_errno(curl))
            {
                throw new Exception(curl_error(curl));
            }

            curl_close(curl);
        }

        // Return?
        if (!one_way)
        {
            return (response);
        }
    }
}

解决PHP7下Session callback expects true/false return value in 问题

升级项目致PHP7遇到Session报警告,既然升级新版本,警告什么的肯定要一起都处理了

session_start(): Session callback expects true/false return value in

经过跟踪排查,发现是到session_start这一步遇到问题,session_start()使用你之前通过session_set_save_handler(….)方法设置的处理器,现在PHP7要求必须返回true/false,所以你需要检查一下你的这些处理器方法是否准确返回true/false了

session_start() uses any handlers you’ve previously set with session_set_save_handler(…), and that handler (or the individual functions) must return a true/false result to signal success or failure. You have a handler that does not return a true/false result. To recap; the actual error is the implementation of your session handler, not the code that calls session_start().

记录Linux系统邮件mail无法发送问题

很多应用,比如wordpress使用PHP的mail函数,将会调用系统邮件发送功能,这时候邮件无法发送该如何排查呢?

  • 安装sendmail来发送邮件
yum -y install sendmail
  • 启动sendmail
/etc/init.d/sendmail start

可通过ps -ef | grep sendmail,观察是否启动成功

  • 测试邮件发送
echo “mail content”| mail -s 'Test mail' accacc@126.com

如果没收到那么查看一下 /var/spool/clientmqueue下面是否存在未发送邮件,tail -f /var/log/maillog 查看邮件发送日志。

另外查看一下网络访问及25端口有无问题。

解决/var/spool/clientmqueue/文件过多问题并记录Linux下快速列出含文件较多文件夹的内容方法

最近发现sendmail的进程较多,观察了一下,是在消费/var/spool/clientmqueue/下的内容,这里面文件太多ls直接卡死了,那么如何列出来呢?

先后使用了ls、tree、find等命令,效果都不明显。

最后使用locate成功解决~

sudo locate clientmqueue > ~/1.txt

观察了一下至少有600多万个文件。。。

当你使用简单的sendmail发邮件的时候, 或者系统默认要发一些邮件(比如cron发的邮件)的时候,首先会把邮件拷贝到这个目录里,然后等待MTA(mail transfer agent) 来处理,MTA做的事情通常是把这个目录中的邮件弄到/var/spool/mqueue里,然后再发送到真正的目的地。出现/var/spool /clientmqueue/非常大的情况通常因为没有合适的MTA发送邮件,就都积累在这里了,假如这里的邮件并不是你需要的,你可以简单的删掉他们。

根据需要发邮件的内容来看,解决这块主要两点:

1、针对crontab一些输出之类的其会发邮件,需要定时任务后加上

 >/dev/null 2>&1

2>是重定向错误

2、梳理业务逻辑控制发邮件频次

当然这两个前段时间都处理了一下,这次主要启用的机器有一些之前的邮件,所以删除了

ls /var/spool/clientmqueue | xargs rm -f

xargs的功能:它将输入输出给xargs后面的命令,作为那个命令的参数

如果你感觉删的比较慢想先快速解决,那么先给他mv一下吧,把事情解决了,然后慢慢删

完事后记得重启一下sendmail sudo /etc/init.d/sendmail restart

Spring Tools报IOException: Server returned HTTP response code: 400 for URL记录

Spring Tools准备创建Spring工程时,报

IOException: Server returned HTTP response code: 400 for URL: http://start.spring.io

这是因为IE为系统配置了代理,但是代理服务器没有开,所以优先检查IE->Internet选项->连接->局域网设置,不要勾选代理服务器,如还不行,检查eclipse的代理设置