【代码复现】GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation

发表于 更新于 分类于 阅读次数： Waline： 本文字数： 1.5k 阅读时长 ≈ 5 分钟
复现GIVEPose的代码，并进行测试。

到目前为止跑不起来，很多报错。

安装GIVEPose环境

  1. 修改GIVEPose_env.yml文件：

    1. 第1行：name: dinov2 -> name: givepose；
    2. 第192行：注释- xformers=0.0.21=py39_cu11.8.0_pyt2.0.1；
    3. 第217行：注释- dcnv3==1.1；
    4. 第268行：注释- pydensecrf==1.0rc2。

  2. 创建环境：

    1
    conda env create -f GIVEPose_env.yml

  3. 激活环境：

    1
    conda activate givepose

  4. 安装dcnv3、pydensecrf和detectron2：

    1
    2
    3
    4
    5
    6
    7
    cd network/ops_dcnv3/
    sh make.sh
    cd ../../

    pip3 install git+https://github.com/lucasb-eyer/pydensecrf.git

    python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'

  5. Clone mentian/object-deformnet并安装nn_distance，需要使用该仓库中的代码进行数据处理：

    1
    2
    3
    4
    git clone https://github.com/mentian/object-deformnet.git

    cd lib/nn_distance
    python setup.py install

  6. Clone lolrudy/LaPose，需要使用该仓库中的代码进行数据处理：

    1
    git clone https://github.com/lolrudy/LaPose.git

数据集下载

  1. 下载NOCS数据集：

    • http://download.cs.stanford.edu/orion/nocs/camera_composed_depth.zip
    • http://download.cs.stanford.edu/orion/nocs/camera_train.zip
    • http://download.cs.stanford.edu/orion/nocs/camera_val25K.zip
    • http://download.cs.stanford.edu/orion/nocs/gts.zip
    • http://download.cs.stanford.edu/orion/nocs/obj_models.zip
    • http://download.cs.stanford.edu/orion/nocs/real_test.zip
    • http://download.cs.stanford.edu/orion/nocs/real_train.zip

  2. 下载NOCS数据集中缺失的物体模型（From: https://github.com/mentian/object-deformnet/issues/3#issuecomment-698858332）：

    • https://drive.google.com/file/d/1rWkxEVJJh_kWIqxudn_i6sJ-hhd0E7TV/

  3. 下载Gorilla-Lab-SCUT/DualPoseNet的分割结果（LaPose使用该分割结果对数据进行预处理）：

    • https://drive.google.com/file/d/1RwAbFWw2ITX9mXzLUEBjPy_g-MNdyHET/

  4. 下载使用mentian/object-deformnet进行数据预处理时用到的数据到仓库文件夹中：

    • https://drive.google.com/file/d/1p72NdY4Bie_sra9U8zoUNI4fTrQZdbnc/

  5. 下载GIVEPose训练时使用的IVFC map：

    • https://drive.google.com/file/d/1kF8ck7EM9mnz6jyySncVUxzXgRxHH9e-/

数据集处理

mentian/object-deformnet

请参考https://blog.032802.xyz/code-running/Leeiieeo_AG-Pose.html#mentianobject-deformnet的处理过程，这里不再重复

lolrudy/LaPose

遵循lolrudy/LaPose的处理方式，首先解压上述文件，并组织如下，注意，data文件夹位于LaPose文件夹下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
data
├── camera -> /data1/dataset/nocs/tanmx/lapose/camera
│   ├── train
│   └── val
├── camera_full_depths -> /data1/dataset/nocs/tanmx/lapose/camera_full_depths/
│   ├── train
│   └── val
├── deformnet_eval -> /data1/dataset/nocs/tanmx/lapose/deformnet_eval
│   ├── camera
│   ├── mrcnn_results
│   ├── nocs_results
│   └── real
├── gts -> /data1/dataset/nocs/tanmx/lapose/gts/
│   ├── real_test
│   └── val
├── obj_models -> /data1/dataset/nocs/tanmx/lapose/obj_models/
│   ├── camera_train.pkl
│   ├── camera_val.pkl
│   ├── mug_meta.pkl
│   ├── real_test
│   ├── real_test.pkl
│   ├── real_train
│   ├── real_train.pkl
│   ├── train
│   └── val
├── real -> /data1/dataset/nocs/tanmx/lapose/real/
│   ├── test
│   └── train
└── segmentation_results -> /data1/dataset/nocs/tanmx/lapose/data/segmentation_results/
    ├── CAMERA25
    └── REAL275

25 directories, 5 files
1
2
3
4
5
6
7
8
9
10
11
12
13
$ ls -l data/obj_models/
total 31520
drwxr-xr-x  6 tanmx tanmx     4096 May  3 12:11 ./
drwxrwxr-x  9 tanmx tanmx     4096 May  3 12:08 ../
-rw-rw-r--  1 tanmx tanmx 26695895 May  3 12:11 camera_train.pkl
-rw-rw-r--  1 tanmx tanmx  4634293 May  3 12:11 camera_val.pkl
-rw-rw-r--  1 tanmx tanmx    23197 May  3 12:11 mug_meta.pkl
drwxrwxr-x  2 tanmx tanmx     4096 Sep 27  2019 real_test/
-rw-rw-r--  1 tanmx tanmx   443600 May  3 12:11 real_test.pkl
drwxrwxr-x  2 tanmx tanmx     4096 Sep 27  2019 real_train/
-rw-rw-r--  1 tanmx tanmx   443584 May  3 12:11 real_train.pkl
drwxr-xr-x 12 tanmx tanmx     4096 Sep 27  2019 train/
drwxr-xr-x 12 tanmx tanmx     4096 Sep 27  2019 val/

按顺序执行：

1
2
python prepare_data/pose_data.py
python prepare_data/shape_data.py

执行python prepare_data/pose_data.py后目录变为：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
data
├── camera -> /data1/dataset/nocs/tanmx/lapose/camera
│   ├── train
│   ├── train_list_all.txt
│   ├── train_list.txt
│   ├── val
│   ├── val_list_all.txt
│   └── val_list.txt
├── camera_full_depths -> /data1/dataset/nocs/tanmx/lapose/camera_full_depths/
│   ├── train
│   └── val
├── deformnet_eval -> /data1/dataset/nocs/tanmx/lapose/deformnet_eval
│   ├── camera
│   ├── mrcnn_results
│   ├── nocs_results
│   └── real
├── gts -> /data1/dataset/nocs/tanmx/lapose/gts/
│   ├── real_test
│   └── val
├── obj_models -> /data1/dataset/nocs/tanmx/lapose/obj_models/
│   ├── camera_train.pkl
│   ├── camera_val.pkl
│   ├── mug_meta.pkl
│   ├── real_test
│   ├── real_test.pkl
│   ├── real_train
│   ├── real_train.pkl
│   ├── train
│   └── val
├── real -> /data1/dataset/nocs/tanmx/lapose/real/
│   ├── test
│   ├── test_list_all.txt
│   ├── test_list.txt
│   ├── train
│   ├── train_list_all.txt
│   └── train_list.txt
└── segmentation_results -> /data1/dataset/nocs/tanmx/lapose/data/segmentation_results/
    ├── CAMERA25
    └── REAL275

25 directories, 13 files
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
$ ls -l data/camera/
total 9192
drwxrwxr-x     4 tanmx tanmx    4096 May  4 05:44 ./
drwxrwxr-x     9 tanmx tanmx    4096 May  3 12:08 ../
drwxr-xr-x 27502 tanmx tanmx  577536 Jun 14  2019 train/
-rw-rw-r--     1 tanmx tanmx 4675000 May  3 12:13 train_list_all.txt
-rw-rw-r--     1 tanmx tanmx 3403978 May  4 03:26 train_list.txt
drwxr-xr-x  2502 tanmx tanmx   69632 Jun 14  2019 val/
-rw-rw-r--     1 tanmx tanmx  375000 May  3 12:13 val_list_all.txt
-rw-rw-r--     1 tanmx tanmx  288300 May  4 05:44 val_list.txt

$ ls -l data/real/
total 288
drwxrwxr-x 4 tanmx tanmx  4096 May  4 06:02 ./
drwxrwxr-x 9 tanmx tanmx  4096 May  3 12:08 ../
drwxrwxr-x 8 tanmx tanmx  4096 Nov 13  2018 test/
-rw-rw-r-- 1 tanmx tanmx 49572 May  3 12:13 test_list_all.txt
-rw-rw-r-- 1 tanmx tanmx 49572 May  4 06:02 test_list.txt
drwxrwxr-x 9 tanmx tanmx  4096 Jun 14  2019 train/
-rw-rw-r-- 1 tanmx tanmx 82042 May  3 12:13 train_list_all.txt
-rw-rw-r-- 1 tanmx tanmx 82042 May  4 04:51 train_list.txt

$ ls -l data/obj_models/
total 31520
drwxr-xr-x  6 tanmx tanmx     4096 May  3 12:11 ./
drwxrwxr-x  9 tanmx tanmx     4096 May  3 12:08 ../
-rw-rw-r--  1 tanmx tanmx 26695895 May  3 12:11 camera_train.pkl
-rw-rw-r--  1 tanmx tanmx  4634293 May  3 12:11 camera_val.pkl
-rw-rw-r--  1 tanmx tanmx    23197 May  3 12:11 mug_meta.pkl
drwxrwxr-x  2 tanmx tanmx     4096 Sep 27  2019 real_test/
-rw-rw-r--  1 tanmx tanmx   443600 May  3 12:11 real_test.pkl
drwxrwxr-x  2 tanmx tanmx     4096 Sep 27  2019 real_train/
-rw-rw-r--  1 tanmx tanmx   443584 May  3 12:11 real_train.pkl
drwxr-xr-x 12 tanmx tanmx     4096 Sep 27  2019 train/
drwxr-xr-x 12 tanmx tanmx     4096 Sep 27  2019 val/

执行python prepare_data/shape_data.py后目录变为：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
data
├── camera -> /data1/dataset/nocs/tanmx/lapose/camera
│   ├── train
│   ├── train_list_all.txt
│   ├── train_list.txt
│   ├── val
│   ├── val_list_all.txt
│   └── val_list.txt
├── camera_full_depths -> /data1/dataset/nocs/tanmx/lapose/camera_full_depths/
│   ├── train
│   └── val
├── deformnet_eval -> /data1/dataset/nocs/tanmx/lapose/deformnet_eval
│   ├── camera
│   ├── mrcnn_results
│   ├── nocs_results
│   └── real
├── gts -> /data1/dataset/nocs/tanmx/lapose/gts/
│   ├── real_test
│   └── val
├── obj_models -> /data1/dataset/nocs/tanmx/lapose/obj_models/
│   ├── camera_train.pkl
│   ├── camera_val.pkl
│   ├── mug_meta.pkl
│   ├── real_test
│   ├── real_test.pkl
│   ├── real_train
│   ├── real_train.pkl
│   ├── ShapeNetCore_2048.h5
│   ├── ShapeNetCore_4096.h5
│   ├── train
│   └── val
├── real -> /data1/dataset/nocs/tanmx/lapose/real/
│   ├── test
│   ├── test_list_all.txt
│   ├── test_list.txt
│   ├── train
│   ├── train_list_all.txt
│   └── train_list.txt
└── segmentation_results -> /data1/dataset/nocs/tanmx/lapose/data/segmentation_results/
    ├── CAMERA25
    └── REAL275

25 directories, 15 files
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
$ ls -l data/camera/
total 9192
drwxrwxr-x     4 tanmx tanmx    4096 May  4 05:44 ./
drwxrwxr-x     9 tanmx tanmx    4096 May  3 12:08 ../
drwxr-xr-x 27502 tanmx tanmx  577536 Jun 14  2019 train/
-rw-rw-r--     1 tanmx tanmx 4675000 May  3 12:13 train_list_all.txt
-rw-rw-r--     1 tanmx tanmx 3403978 May  4 03:26 train_list.txt
drwxr-xr-x  2502 tanmx tanmx   69632 Jun 14  2019 val/
-rw-rw-r--     1 tanmx tanmx  375000 May  3 12:13 val_list_all.txt
-rw-rw-r--     1 tanmx tanmx  288300 May  4 05:44 val_list.txt

$ ls -l data/real/
total 288
drwxrwxr-x 4 tanmx tanmx  4096 May  4 06:02 ./
drwxrwxr-x 9 tanmx tanmx  4096 May  3 12:08 ../
drwxrwxr-x 8 tanmx tanmx  4096 Nov 13  2018 test/
-rw-rw-r-- 1 tanmx tanmx 49572 May  3 12:13 test_list_all.txt
-rw-rw-r-- 1 tanmx tanmx 49572 May  4 06:02 test_list.txt
drwxrwxr-x 9 tanmx tanmx  4096 Jun 14  2019 train/
-rw-rw-r-- 1 tanmx tanmx 82042 May  3 12:13 train_list_all.txt
-rw-rw-r-- 1 tanmx tanmx 82042 May  4 04:51 train_list.txt

$ ls -l data/obj_models/
total 116064
drwxr-xr-x  6 tanmx tanmx     4096 May  4 11:15 ./
drwxrwxr-x  9 tanmx tanmx     4096 May  3 12:08 ../
-rw-rw-r--  1 tanmx tanmx 26695895 May  4 10:28 camera_train.pkl
-rw-rw-r--  1 tanmx tanmx  4634293 May  4 10:32 camera_val.pkl
-rw-rw-r--  1 tanmx tanmx    23197 May  4 10:32 mug_meta.pkl
drwxrwxr-x  2 tanmx tanmx     4096 Sep 27  2019 real_test/
-rw-rw-r--  1 tanmx tanmx   443600 May  4 10:32 real_test.pkl
drwxrwxr-x  2 tanmx tanmx     4096 Sep 27  2019 real_train/
-rw-rw-r--  1 tanmx tanmx   443584 May  4 10:32 real_train.pkl
-rw-rw-r--  1 tanmx tanmx 28951728 May  4 11:15 ShapeNetCore_2048.h5
-rw-rw-r--  1 tanmx tanmx 57616135 May  4 10:35 ShapeNetCore_4096.h5
drwxr-xr-x 12 tanmx tanmx     4096 Sep 27  2019 train/
drwxr-xr-x 12 tanmx tanmx     4096 Sep 27  2019 val/

运行代码

将数据集组织如下：