THuman Dataset preprocess

[glorioushonor]

Good job! When I run the command of "python -m scripts.render_batch -debug -headless", it gives an error as following:

Star /t Rendering thuman2 with 36 views, 512x512 size.
Output dir: ./debug/thuman2_36views
Rendering types: ['light', 'normal', 'depth']
  0%|                                                                                                                                                                                 | 0/2 [00:00<?, ?it/s]libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
Compilation failure for vertex shader:
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
Compilation failure for fragment shader:
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
Linker failure: 
b'error: linking with uncompiled/unspecialized shadererror: linking with uncompiled/unspecialized shader'
  0%|                                                                                                                                                                                 | 0/2 [00:03<?, ?it/s]
Traceback (most recent call last):
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/media/linjie/2473F3E9F7F20D6D/Code/ICON/scripts/render_batch.py", line 254, in <module>
    for _ in tqdm(
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fe4a0bf7a30>'. Reason: 'ValueError('ctypes objects containing pointers cannot be pickled')'

How can I fix this?

[YuliangXiu]

Have you ever tried to Google the solution?

Such as https://stackoverflow.com/questions/52592309/glsl-3-30-is-not-supported-when-compiling-glsl-shader-on-ubuntu-18-04

[glorioushonor]

你有没有尝试过谷歌解决方案？

如 https://stackoverflow.com/questions/52592309/glsl-3-30-is-not-supported-when-compiling-glsl-shader-on-ubuntu-18-04

Thanks for your reply.
Yes, I searched for this problem before, and have run "export MESA_GL_VERSION_OVERRIED=3.3" to solve "b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n' ", but there are still errors, which seem to be related to opengl, But I think my computer has installed pyopengl==3.1.1a1 and Pyopengl-Accelerate ==3.1.6 correctly. Could you give me some help, please?

(ICON) linjie@linjie-Z390-GAMING-X:/media/linjie/2473F3E9F7F20D6D/Code/ICON$ python -m scripts.render_batch -headless -out_dir data/
Start Rendering thuman2 with 36 views, 512x512 size.
Output dir: data//thuman2_36views
Rendering types: ['light', 'normal']
  0%|                                                                                                                                                                               | 0/526 [00:00<?, ?it/s]libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
  0%|                                                                                                                                                                               | 0/526 [00:46<?, ?it/s]
Traceback (most recent call last):
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/media/linjie/2473F3E9F7F20D6D/Code/ICON/scripts/render_batch.py", line 254, in <module>
    for _ in tqdm(
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fb463cc16a0>'. Reason: 'ValueError('ctypes objects containing pointers cannot be pickled')'

[glorioushonor]

你有没有尝试过谷歌解决方案？

如 https://stackoverflow.com/questions/52592309/glsl-3-30-is-not-supported-when-compiling-glsl-shader-on-ubuntu-18-04

I'm sorry to bother you again. Aiming at solving these problems, I've been browsing all of Google and issues in your GitHub, but I still can't fix the problem which seem to be related to opengl and multiprocessing. I can't seem to go on, and it seems that a lot of people have problems here. Could you provide the synthetic dataset for training, or give some suggesions please.

[YuliangXiu]

You could use the old rendering code, see the old dataset.md and old scripits

[glorioushonor]

You could use the old rendering code, see the old dataset.md and old scripits

Thank you very much! The older version of the script worked fine, and I was able to fix some minor problems.
But I got the libEGL warning: DRI2: failed to authenticate warning, but I did get the output under debug/, not sure if it had any bad effects.
And to run the old script correctly, I change the ms_rate=16 to ms_rate=1.0 in line 115 of render_single.py. I already know that this setting is bad for anti-aliasing, but my computer can only achieve ms_rate=1.0, otherwise the program will report an error. Should I switch to a higher performance GPU? Is the second script running long? I've been running it for half a day (1080 Ti).
BTW, congratulations on your paper ECON being accepted by CVPR2023.

[glorioushonor]

You could use the old rendering code, see the old dataset.md and old scripits

I'm sorry to bother you again.
I found the rendering process time-consuming. After browsing the script file, I find that two Gpus are used to run the script. I wonder if my understanding is correct.

[YuliangXiu]

One GPU is enough, if it's too slow, maybe #62 (comment) is related to your issue.

[glorioushonor]

And to run the old script correctly, I change the ms_rate=16 to ms_rate=1.0 in line 115 of render_single.py. I already know that this setting is bad for anti-aliasing, but my computer can only achieve ms_rate=1.0, otherwise the program will report an error. Should I switch to a higher performance GPU?

Thanks to your kind reply.
I think it's working correctly. I used a 1080Ti GPU and the script has been running for 24 hours and has rendered about half of the data set. How long does the rendering process take on the Quadro RTX 5000 which you used?
In addition, to run the old script correctly, I change the ms_rate=16 to ms_rate=1.0 in line 115 of render_single.py. I already know that this setting is bad for anti-aliasing, but my computer can only achieve ms_rate=1.0, otherwise the program will report an error.
To sum up, I'm going to use a single 3090 for the render and rerender process. Am I right?

[YuliangXiu]

It's kind of slow, 36 views per scan only takes me a few hours for rendering. You can use cprofile to check the performance bottleneck. If the PBR generation takes too long, see #62 (comment) for the solution.

I guess the ms_rate=1.0 won't affect your final results too much.

[glorioushonor]

You could use the old rendering code, see the old dataset.md and old scripits

Sorry to bother you again, and thank you for your help before. I previously rendered the THuman2.0 dataset by running the old version of the script, but the preprocessed data cannot aligned to the smplx mesh. The results are as follows:

I noticed by browsing GitHub that you suggested a new script to solve this problem. But now I can't run through the new script, is there any way to correct the old script to make the result correct instead of using the new script? Could you please give me some advice, thank you.

[MalignusCN]

Good job! When I run the command of "python -m scripts.render_batch -debug -headless", it gives an error as following:

Star /t Rendering thuman2 with 36 views, 512x512 size.
Output dir: ./debug/thuman2_36views
Rendering types: ['light', 'normal', 'depth']
  0%|                                                                                                                                                                                 | 0/2 [00:00<?, ?it/s]libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
Compilation failure for vertex shader:
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
Compilation failure for fragment shader:
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
Linker failure: 
b'error: linking with uncompiled/unspecialized shadererror: linking with uncompiled/unspecialized shader'
  0%|                                                                                                                                                                                 | 0/2 [00:03<?, ?it/s]
Traceback (most recent call last):
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/media/linjie/2473F3E9F7F20D6D/Code/ICON/scripts/render_batch.py", line 254, in <module>
    for _ in tqdm(
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fe4a0bf7a30>'. Reason: 'ValueError('ctypes objects containing pointers cannot be pickled')'

How can I fix this?

I've found the same issue here, directly use
export MESA_GL_VERSION_OVERRIDE=3.3
doesn't useful for me, but with os.environ is useful

[glorioushonor]

Good job! When I run the command of "python -m scripts.render_batch -debug -headless", it gives an error as following:

Star /t Rendering thuman2 with 36 views, 512x512 size.
Output dir: ./debug/thuman2_36views
Rendering types: ['light', 'normal', 'depth']
  0%|                                                                                                                                                                                 | 0/2 [00:00<?, ?it/s]libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
Compilation failure for vertex shader:
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
Compilation failure for fragment shader:
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
Linker failure: 
b'error: linking with uncompiled/unspecialized shadererror: linking with uncompiled/unspecialized shader'
  0%|                                                                                                                                                                                 | 0/2 [00:03<?, ?it/s]
Traceback (most recent call last):
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/media/linjie/2473F3E9F7F20D6D/Code/ICON/scripts/render_batch.py", line 254, in <module>
    for _ in tqdm(
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fe4a0bf7a30>'. Reason: 'ValueError('ctypes objects containing pointers cannot be pickled')'

How can I fix this?

I've found the same issue here, directly use export MESA_GL_VERSION_OVERRIDE=3.3 doesn't useful for me, but with os.environ is useful

Hello, thanks for your kind sharing. Whether you use os.environ to solve
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
or another error
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fe4a0bf7a30>'. Reason: 'ValueError('ctypes objects containing pointers cannot be pickled')'
I am still looking for a solution to the second error. Could you please give me some help.

[yangqinhui0423]

Good job! When I run the command of "python -m scripts.render_batch -debug -headless", it gives an error as following:

Star /t Rendering thuman2 with 36 views, 512x512 size.
Output dir: ./debug/thuman2_36views
Rendering types: ['light', 'normal', 'depth']
  0%|                                                                                                                                                                                 | 0/2 [00:00<?, ?it/s]libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to authenticate
Compilation failure for vertex shader:
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
Compilation failure for fragment shader:
b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n'
Linker failure: 
b'error: linking with uncompiled/unspecialized shadererror: linking with uncompiled/unspecialized shader'
  0%|                                                                                                                                                                                 | 0/2 [00:03<?, ?it/s]
Traceback (most recent call last):
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/media/linjie/2473F3E9F7F20D6D/Code/ICON/scripts/render_batch.py", line 254, in <module>
    for _ in tqdm(
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/home/linjie/anaconda3/envs/ICON/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fe4a0bf7a30>'. Reason: 'ValueError('ctypes objects containing pointers cannot be pickled')'

How can I fix this?

I've found the same issue here, directly use export MESA_GL_VERSION_OVERRIDE=3.3 doesn't useful for me, but with os.environ is useful

Hello, thanks for your kind sharing. Whether you use os.environ to solve b'0:1(10): error: GLSL 3.30 is not supported. Supported versions are: 1.10, 1.20, 1.30, 1.40, 1.00 ES, and 3.00 ES\n' or another error multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fe4a0bf7a30>'. Reason: 'ValueError('ctypes objects containing pointers cannot be pickled')' I am still looking for a solution to the second error. Could you please give me some help.

Hey, I want to know if you have solved the second error. I also met the same error and have tried all the solutions above but it's not solved.
My error is as follows:
_(icon) ciciyang@DESKTOP-RVT20TR:~/code/ICON$ python -m scripts.render_batch -debug -headless
Start Rendering thuman2 with 36 views, 512x512 size.
Output dir: ./debug/thuman2_36views
Rendering types: ['light', 'normal', 'depth']
0%| | 0/2 [00:04<?, ?it/s]Traceback (most recent call last):
File "/home/ciciyang/miniconda3/envs/icon/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/ciciyang/miniconda3/envs/icon/lib/python3.8/runpy.py", line 87, in run_code
exec(code, run_globals)
File "/home/ciciyang/code/ICON/scripts/render_batch.py", line 254, in
for _ in tqdm(
File "/home/ciciyang/miniconda3/envs/icon/lib/python3.8/site-packages/tqdm/std.py", line 1180, in iter
for obj in iterable:
File "/home/ciciyang/miniconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 868, in next
raise value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f8c12b3dfa0>'. Reason: 'ValueError('ctypes objects containing pointers cannot be pickled')'