非视距成像|Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization 代码运行记录非视距成像|Code

Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization Miika Aittala, Prafull Sharma, Lukas Murmann, Adam Yedidia, Gregory Wornell, William T. Freeman, and Frédo Durand 【非视距成像|Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization 代码运行记录】
论文Github地址：https://github.com/prafull7/compmirrors
论文原文：https://arxiv.org/abs/1912.02314
论文Project: http://compmirrors.csail.mit.edu/

Table of Contents
矩阵分解部分(factorization_1d.py)
Error1 AttributeError: module 'scipy.misc' has no attribute 'imresize'
重建部分
数据集
参数设置
Error2 ImportError: cannot import name 'imsave'
Error3 ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url
Error4 IndexError: list index out of range
总结

矩阵分解部分(factorization_1d.py) Error1 AttributeError: module 'scipy.misc' has no attribute 'imresize'

Traceback (most recent call last):
File "factorization_1d.py", line 74, in
T0 = scipy.misc.imresize(T0, [h, q]).astype(float) / 255
AttributeError: module 'scipy.misc' has no attribute 'imresize'

解决方案：https://blog.csdn.net/WUDIxi/article/details/100060055
更改代码：

# T0 = scipy.misc.imresize(T0, [h, q]).astype(float) / 255 # L0 = scipy.misc.imresize(L0, [q, w]).astype(float) / 255 T0 = resize(T0, [h, q]).astype(float) / 255 L0 = resize(L0, [q, w]).astype(float) / 255

即可

至此，single picture的矩阵分解部分代码调通。明天再调Reconstruction部分的代码。
重建部分数据集 Reconstruction部分的数据从http://compmirrors.csail.mit.edu/ 下载；
并按照说明处理：

Place the folder in './data/light_transport/' to be consistent with instructions on github.

参数设置根据github的instruction，不妨将参数按如下进行设置：

parser.add_argument('-d', '--data_dir', type=str, default=os.environ.get('FACTORIZE_DATA_DIR', './data/light_transport/'), help='Location for data directory') parser.add_argument('-o', '--out_dir', type=str, default=os.environ.get('FACTORIZE_OUT_DIR', './result/'), help='Location for output directory') parser.add_argument('-f', '--folder', type=str, default='dots_data', help='Name of the folder') parser.add_argument('-ds', '--dataset', type=str, default='dots_sequence', help='Name of the dataset') parser.add_argument('-s', '--seq_name', type=str, default='dots', help='Name of the sequence')

同时注意visdom的端口，为了防止和别人冲突改为9103

parser.add_argument('-vis_p', '--visdom_port', type=int, default=9103, help='Port of the visdom server')

并通过下面代码开启端口：

python -m visdom.server -port 9103

同时运行代码可改为：

python factorization_light_transport.py -dev DEVICE_NUMBER

Error2 ImportError: cannot import name 'imsave'

ImportError: cannot import name 'imsave'

参考：https://blog.csdn.net/chao2016/article/details/77971577
并且scipy版本不能达到1.5.0，因此通过conda降级到1.2.1：
conda install scipy==1.2.1
问题解决

Error3 ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url

Traceback (most recent call last): File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection raise err File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refusedDuring handling of the above exception, another exception occurred:Traceback (most recent call last): File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request conn.request(method, url, **httplib_request_kw) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 1026, in _send_output self.send(msg) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 964, in send self.connect() File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect conn = self._new_conn() File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refusedDuring handling of the above exception, another exception occurred:Traceback (most recent call last): File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 725, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/util/retry.py", line 439, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /env/2020-08-12-1051--dots_dots_sequence_gpu01_0 (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))During handling of the above exception, another exception occurred:Traceback (most recent call last): File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/visdom/__init__.py", line 484, in _send data=https://www.it610.com/article/json.dumps(msg), File"/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/api.py", line 119, in post return request('post', url, data=https://www.it610.com/article/data, json=json, **kwargs) File"/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /env/2020-08-12-1051--dots_dots_sequence_gpu01_0 (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',)) Device ID: 0 Traceback (most recent call last): File "factorization_light_transport.py", line 101, in frames_dict = json.load(open(os.path.join(args.data_dir, args.folder, "frames.txt"),'r')) File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/posixpath.py", line 80, in join a = os.fspath(a) TypeError: expected str, bytes or os.PathLike object, not NoneType

前面所说的port没有改...开的是9103，用的却是没有开的8097，故出现错误。

Error4 IndexError: list index out of range

Traceback (most recent call last): File "factorization_light_transport.py", line 488, in factorize() File "factorization_light_transport.py", line 147, in factorize Z, overexp_mask, gt_imgs = load_observations(data_dir, folder_name, dataset, seq_name, skip_frames, n_downsample=args.num_downsample) File "/share2/home/ruixu/DeepLearningLCT/DeblurGAN-master/compmirrors-master/compmirrors-master/data_loading.py", line 117, in load_observations blacks, blacks_oe = load_pgm(os.path.join(data_dir, folder_name, dataset + "_reflector"), frames_dict['black'][0], frames_dict['black'][1], n_downsample) File "/share2/home/ruixu/DeepLearningLCT/DeblurGAN-master/compmirrors-master/compmirrors-master/data_loading.py", line 51, in load_pgm img = cv2.imread(files[i], cv2.IMREAD_UNCHANGED) IndexError: list index out of range

经过调试可以发现，data_loading.py中load_pgm的files是None，即path不对，从而导致了上面的错误。
修改方法：
step1: 将data_loading.py中load_observations代码的的116-120行中的reflector后面加上/，改为：

# Read black frames if available if 'black' in frames_dict:# {'black': [0, 99], 'impulse': [100, 675], 'black2': [676, 707], 'disc': [708, 4290]} blacks, blacks_oe = load_pgm(os.path.join(data_dir, folder_name, dataset + "_reflector/"), frames_dict['black'][0], frames_dict['black'][1], n_downsample)imgs, imgs_oe = load_pgm(os.path.join(data_dir, folder_name, dataset + "_reflector/"), frames_dict[seq_name][0], frames_dict[seq_name][1], n_downsample)

读取ft图片时同理：

gt_imgs = load_gt(os.path.join(data_dir, folder_name, dataset + "_target/"), frames_dict[seq_name][0], frames_dict[seq_name][1], ext="")

step2: 将文件结构改为：

非视距成像|Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization 代码运行记录

文章图片

其中，第一个reflector文件夹中是文件名包含reflector的图像(即相机observe到的图片)；第二个文件夹则是gt.

这里有一个问题，即读取gt时后缀为什么是""？这是因为708正好分割开了target_img的两个部分。
=================================================================================
至此，程序可以正常运行：