sparksql代码执行过于缓慢

最近服务器状态不佳,oracle有时性能较差,运行sparksql向oracle中存数据时运行超级缓慢,开始还特意排查代码找错,最终换了一个oracle数据库做实验发现整个流程在几秒内非常顺利的完成了,可见oracle性能对sparksql执行速度影响巨大,特此记录此次事件。
以下是运行时部分日志,可以看到每执行1/200的进度要30000ms,整个预计要200*30=6000秒才能完毕。。。。。。

2019-03-15 15:11:51 INFOTaskSetManager:54 - Finished task 2.0 in stage 1.0 (TID 3) in 30618 ms on localhost (executor driver) (3/200) 2019-03-15 15:11:51 INFOShuffleBlockFetcherIterator:54 - Getting 0 non-empty blocks out of 1 blocks 2019-03-15 15:11:51 INFOShuffleBlockFetcherIterator:54 - Started 0 remote fetches in 1 ms 2019-03-15 15:12:22 WARNJdbcUtils:66 - Requested isolation level 1 is not supported; falling back to default isolation level 2 2019-03-15 15:12:22 INFOExecutor:54 - Finished task 3.0 in stage 1.0 (TID 4). 2524 bytes result sent to driver 2019-03-15 15:12:22 INFOTaskSetManager:54 - Starting task 4.0 in stage 1.0 (TID 5, localhost, executor driver, partition 4, PROCESS_LOCAL, 7754 bytes) 2019-03-15 15:12:22 INFOExecutor:54 - Running task 4.0 in stage 1.0 (TID 5)

【sparksql代码执行过于缓慢】

    推荐阅读