From 93581ceb798a9707e1356e08cd3e7e7bebf659c4 Mon Sep 17 00:00:00 2001 From: Qingwen Zhang <35365764+Kin-Zhang@users.noreply.github.com> Date: Thu, 8 Feb 2024 17:37:16 +0100 Subject: [PATCH 1/3] Initial commit --- LICENSE | 28 ++++++++++++++++++++++++++++ README.md | 2 ++ 2 files changed, 30 insertions(+) create mode 100644 LICENSE create mode 100644 README.md diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..694508c --- /dev/null +++ b/LICENSE @@ -0,0 +1,28 @@ +BSD 3-Clause License + +Copyright (c) 2024, Robotics, Perception and Learning @KTH + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..7636dec --- /dev/null +++ b/README.md @@ -0,0 +1,2 @@ +# SeFlow +[Under review] SeFlow From eeeea26a885b20be898c19fc4e65a46fc3c41e01 Mon Sep 17 00:00:00 2001 From: Kin Date: Mon, 1 Jul 2024 20:42:08 +0200 Subject: [PATCH 2/3] docs: update README about code structure. --- README.md | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 81 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 7636dec..cdadd1b 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,81 @@ -# SeFlow -[Under review] SeFlow +SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving +--- + +[arXiv coming soon] [poster comming soon] [viceo coming soon] + +We got accepted! I will **update the whole code** around the DDL of the camera-ready version (**Jul'15**). Stay Tunned with us. + +Task: Self-Supervvised Scene Flow Estimation in Autonomous Driving. + +**Scripts** quick view in our scripts (We directly follow our previous work [DeFlow code structure](https://github.com/KTH-RPL/DeFlow)): + +- `dataprocess/extract_*.py` : pre-process data before training to speed up the whole training time. + [Dataset we included now: Argoverse 2 and Waymo. more on the way: Nuscenes, custom data.] + +- `0_process.py`: process data with save dufomap, cluster labels inside file. + +- `1_train.py`: Train the model and get model checkpoints. Pls remember to check the config. + +- `2_eval.py` : Evaluate the model on the validation/test set. And also upload to online leaderboard. + +- `3_vis.py` : For visualization of the results with a video. + + +## 0. Setup + +**Environment**: Same to DeFlow. + + +## 1. Run & Train + +comming soon + +## 2. Evaluation & Upload to Leaderboard + +You can view Wandb dashboard for the training and evaluation results or upload result to online leaderboard. + +Since in training, we save all hyper-parameters and model checkpoints, the only thing you need to do is to specify the checkpoint path. Remember to set the data path correctly also. + +```bash +# downloaded pre-trained weight, or train by yourself +wget TODO +python 2_eval.py checkpoint=/home/kin/seflow_best.ckpt av2_mode=val # it will directly prints all metric +python 2_eval.py checkpoint=/home/kin/seflow_best.ckpt av2_mode=test # it will output the av2_submit.zip for you to submit to leaderboard +``` + + +## 3. Visualization + +We provide a script to visualize the results of the model. You can specify the checkpoint path and the data path to visualize the results. The step is quickly similar to evaluation. + +```bash + +# Then run the command in the terminal: +python tests/scene_flow.py --flow_mode 'deflow_best' --data_dir /home/kin/data/av2/preprocess/sensor/mini +``` + + +## Cite & Acknowledgements + +``` +@article{zhang2024seflow, + author={Zhang, Qingwen and Yang, Yi and Li, Peizheng and Andersson, Olov and Jensfelt, Patric}, + title={SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving}, + journal={arXiv preprint arXiv:TODO}, + year={2024} +} +@article{zhang2024deflow, + author={Zhang, Qingwen and Yang, Yi and Fang, Heng and Geng, Ruoyu and Jensfelt, Patric}, + title={DeFlow: Decoder of Scene Flow Network in Autonomous Driving}, + journal={arXiv preprint arXiv:2401.16122}, + year={2024} +} +``` + +Thanks to RPL member: Li Ling helps review our SeFlow manuscript. +Thanks to [Kyle Vedder (ZeroFlow)](https://github.com/kylevedder), who kindly opened his code including pre-trained weights, and discussed their result with us which helped this work a lot. +This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation and Prosense (2020-02963) funded by Vinnova. +The computations were enabled by the supercomputing resource Berzelius provided by National Supercomputer Centre at Linköping University and the Knut and Alice Wallenberg Foundation, Sweden. + +❤️: [DeFlow](https://github.com/KTH-RPL/DeFlow), [BucketedSceneFlowEval](https://github.com/kylevedder/BucketedSceneFlowEval) + From cb64849f333e90372b81adcdfadf87fb99f5f2f7 Mon Sep 17 00:00:00 2001 From: Qingwen Zhang <35365764+Kin-Zhang@users.noreply.github.com> Date: Mon, 1 Jul 2024 20:42:55 +0200 Subject: [PATCH 3/3] docs: fix typo --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index cdadd1b..c31be02 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving We got accepted! I will **update the whole code** around the DDL of the camera-ready version (**Jul'15**). Stay Tunned with us. -Task: Self-Supervvised Scene Flow Estimation in Autonomous Driving. +Task: Self-Supervised Scene Flow Estimation in Autonomous Driving. **Scripts** quick view in our scripts (We directly follow our previous work [DeFlow code structure](https://github.com/KTH-RPL/DeFlow)): @@ -28,7 +28,7 @@ Task: Self-Supervvised Scene Flow Estimation in Autonomous Driving. ## 1. Run & Train -comming soon +coming soon ## 2. Evaluation & Upload to Leaderboard