https://rubatotree.github.io/blog/tags/%E5%85%B7%E8%BA%AB%E6%99%BA%E8%83%BD/Recent content in 具身智能 on Umbrella Coffeehttps://rubatotree.github.io/blog/images/og-default.pnghttps://rubatotree.github.io/blog/images/og-default.pngHugoen-usSun, 07 Jun 2026 00:00:00 +0800https://rubatotree.github.io/blog/posts/embodied-notes-2/Sun, 07 Jun 2026 00:00:00 +0800https://rubatotree.github.io/blog/posts/embodied-notes-2/<p class="typst-parbreak"></p> <div style="display: grid; place-items: start center;"> <figure> <div style="display: grid; place-items: start center;"><img src="https://rubatotree.github.io/blog/images/embodied-notes-2/teaser.png" alt loading="lazy"></div> <div style="display: grid; place-items: start center;"> <figcaption>图 1 玩音乐的最终归宿是转行具身智能。<a id="loc-1" href="#loc-3" role="doc-biblioref">[1]</a></figcaption> </div> </figure> </div> <p class="typst-parbreak"></p> <p>TODO</p> <div class="toc" style="display: none"><details><summary>Table of Contents</summary><div><nav role="doc-toc"><ol style="list-style-type: none"><li><p></p><ul><li><a href="#loc-2">References</a></li></ul><p></p></li></ol></nav></div></details></div> <section role="doc-bibliography"> <h2 id="loc-2">References</h2> <ul style="list-style-type: none"> <li id="loc-3"><span class="prefix"><a href="#loc-1" role="doc-backlink">[1]</a></span> 山下清悟, “超かぐや姫！.” [Online]. Available: <a href="https://www.netflix.com/title/81756595" target="_blank" rel="noopener noreferrer"><span style="color: #59a4ff;"><span style="text-decoration: underline">https://www.netflix.com/title/81756595</span></span></a></li> <li><span class="prefix">[2]</span> J. Park and H. Kang, “RenderMem: Rendering as Spatial Memory Retrieval.” [Online]. Available: <a href="https://arxiv.org/abs/2603.14669" target="_blank" rel="noopener noreferrer"><span style="color: #59a4ff;"><span style="text-decoration: underline">https://arxiv.org/abs/2603.14669</span></span></a></li> <li><span class="prefix">[3]</span> R. I. C. Muchacho and F. T. Pokorny, “Walk on Spheres for PDE-based Path Planning.” [Online]. Available: <a href="https://arxiv.org/abs/2406.01713" target="_blank" rel="noopener noreferrer"><span style="color: #59a4ff;"><span style="text-decoration: underline">https://arxiv.org/abs/2406.01713</span></span></a></li> <li><span class="prefix">[4]</span> C. Jambon, M. S. Nabizadeh, and M. Konaković Luković, “Walk on Decomposed Subdomains: A Hybrid Monte Carlo–Deterministic Solver for Elliptic PDEs,” <em>ACM Trans. Graph.</em>, vol. 45, no. 4, July 2026, doi: <a href="https://doi.org/10.1145/3811340" target="_blank" rel="noopener noreferrer"><span style="color: #59a4ff;"><span style="text-decoration: underline">10.1145/3811340</span></span></a>.</li> <li><span class="prefix">[5]</span> Heskey0, “具身智能 - 9 个方向讲透 2025-2026 灵巧手智能.” [Online]. Available: <a href="https://zhuanlan.zhihu.com/p/2046746760459171551" target="_blank" rel="noopener noreferrer"><span style="color: #59a4ff;"><span style="text-decoration: underline">https://zhuanlan.zhihu.com/p/2046746760459171551</span></span></a></li> </ul> </section>https://rubatotree.github.io/blog/posts/embodied-notes-1-robosplat/Thu, 04 Jun 2026 00:00:00 +0800https://rubatotree.github.io/blog/posts/embodied-notes-1-robosplat/<p>这周组会上导师提到现在做 Rendering 已经很难有价值和影响力了，也很难找到好的课题，应该 all in 具身智能。感觉自己从一个转型阵痛期的组跳进了另一个转型阵痛期的组。不过转型是正确的，具身方向的确好发有影响力的论文，就业前景也比传统图形学好。虽然私心比较喜欢简洁优雅但没用的图形学，但确实应该现实一点。我们该思考一下怎样把已有的图形学、渲染经验搬到具身智能的应用上。</p> <p>组里同学分享的 RoboSplat <a id="loc-1" href="#loc-10" role="doc-biblioref">[1]</a> 一文确实相当契合 CG for embodied 的愿景，因此我们从这篇论文开始了解我们能够做什么。</p> <h2 id="loc-2">论文概要</h2> <p>本文的目标是通过输入场景的多视角 RGB 图片和<strong>单次</strong>示教样本（Expert Demonstration，人工操作机械臂完成一次任务，得到每帧的机械臂状态数据和单监督视角 RGB 图像序列）训练 VLA 模型泛化完成不同干扰场景下的同一任务。本文试图解决的干扰场景分为物体位姿移动(Object Pose)、监督相机移动(Camera View)、光照条件改变(Lighting)、物体种类改变(Object Type)、周边环境改变(Appearance)、机械臂外观改变(Cross Embodiment)六种。</p> <p class="typst-parbreak"></p> <div style="display: grid; place-items: start center;"> <figure> <div style="display: grid; place-items: start center;"><img src="https://rubatotree.github.io/blog/images/embodied-notes-1-robosplat/robosplat-teaser.png" alt loading="lazy"></div> <div style="display: grid; place-items: start center;"> <figcaption>图 1 论文 teaser，展示了 RoboSplat 的训练过程。</figcaption> </div> </figure> </div> <p class="typst-parbreak"></p> <p>本文做到泛化的具体方法则是做<strong>数据增强</strong>，从单次示教样本中根据这几种干扰情况生成上千条增强样本 (augmented demonstrations)。以往生成增强样本的思路是直接在 2D 图像空间做生成式编辑，而本文考虑将多视角图片训练成 3DGS 场景（并在语义上分离场景、物体、机械臂，给机械臂绑骨），去在 3DGS 场景中做编辑模拟这些干扰，再合成回样本图像序列。这样做的好处是保证了图像之间场景的一致性，且不会有抖动等情况出现，因此能准确指导 VLA 模型学习到正确的操作策略。</p> <p>本文对这几种任务做了测试：</p> <p></p> <ul> <li> <p>Pick Object（抓取物体）：机械臂需要抓起放置在 30cm*40cm 范围桌面上的目标物体。</p> <p></p> <ul> <li>难点： 测试基本的空间 3D 定位与抓取能力。</li> </ul> <p></p> </li> <li> <p>Close Drawer（关闭抽屉）：机械臂需要推关一个抽屉。抽屉的位置在 15cm*40cm 范围内随机变化，且抽屉在 Z 轴方向的旋转角度在 [-pi/8, pi/8] 之间随机偏转。</p>