Surface Normals & Properties Estimation on Point Sets

Surface Normals & Properties Estimation on Point Sets

1.      Introduction

Point sets or point clouds (PC) are collection of points that represent a 3D shape. It is an inherently discontinuous primitive compared to triangles and volumetric for mesh representation. Often PC’s are acquired without normals from 3D scanners for engineering, medical, or visual effects applications. Point Normals are crucial for shading, determining connectivity among neighboring points, forming surfaces, and geometry processing operations such as denoising and resampling. The purpose of this project is to implement normals estimation techniques for point clouds, and explore the relevant surface parameters.

Go to Point Cloud posts

NB: Posts are excerpts from ongoing thesis reports.

Gundam Breaker 2 First Impressions

Apparently the blog lay dormant for two years. Except work NDA, I should allow myself to write anything else.

Gundam Breaker 2 was released 2014/12/18 on ps3/VITA, with JP/CH choices. Took awhile to ship the Chinese version (Asia version on to far away New Zealand. Chinese includes Japanese voice and optional subtitles in Taiwanese or Hong Kong formats. Voice acting and translations are fairly well done.

I’ve checked PSN from several countries, and there’s more description on the JP PSN:

The AI comrades are voiced by Hikasa Yoko as Leia and Katsuyuki Konichi as Kalevi. Both are quite senior seiyuus and do a good job. Since gameplay is fast paced, short small talk ends up repetitive. Dialog bubbles appear from their mech during fights and is a bit amusing tho hard to read as the bubbles flys along them. If you bump or shoot them they’ll say “mech is damaged!”. It would be nice to vary dialog so they grow angry at the noob pilot who friendly fires. But it’s a mech game, so don’t expect Last of Us (which I finished before this). Abe Atsushi as Shouma appears later (I just started GB2). Since he’s famed for Kamijou Touma he may be an important character, right fist included.

Coupons: I played for awhile (most time spent on tweaking metallic/specular shaders for each mech part), but didn’t try them on NZ PSN. Gamefaq suggests Asia or JP region PSN’s to get future DLC’s. So I created a Taiwan PSN (same as HK, you can choose English during signup on the website, and use any address). Judging how GB1 had many DLC’s (in TW but not NZ PSN), I made the switch and replayed some missions and re-tweaked each shader again. It’s my first gundam game and I’m fairly new to the franchise so it didn’t feel too repetitive.
There are 2 codes. Enter at bottom of PSN store in ps3. Go to the GB2 hangar, choose the right-most booth (gundam online store). Gift tab shows the coupon items. Download each one. Now they’re available in the mech part lists. Most parts are high level so unusable before playing more, but a few available are very high specced. A few upgrades make initial missions overly easy, so I am putting the off till later.

Difficulty: each mission shows a difficulty bar. But I couldn’t find any difficulty setting, so the parts you use may have a drastic effect on challenge. The Challenge Mode (square button to Gundam Cafe) is notably harder: 5-10 minutes of fighting swarms of gundams where each takes significantly longer to defeat. I started dying 1-2 times a minute, since they easily swarm you and during the chaos your combat parts may get severed. It’s quite amusing since it takes awhile in the chaotic scene to notice button mashing is not attacking because your arm is on the far end of the table. Find your arm/hand/shield etc and hold R1 to absorb it back.

Healing: R1+triangle heals all friendlies, with ~1x seconds cooldown. Up arrow has a heal tool (20 uses per mission), but couldn’t figure how to use it yet. Friendlies have no lifebars though.

Story & Screencaps: tutorial mission, story/dialogs I briefly translated. Exposition is on the JP website, and I can’t quite read Japanese. (bad translation follows…) “It does stress GB1 didn’t have story, and GB2 they emphasize a story & world. In year 2024, there’s a Gundam World First event opening, and an attraction called Gundam Battle Simulator 2.0 (also mentioned in game). The story is an epic about the Earthling Army & Colonial Allies (whatever matches the gundam factions).”

Real-time Editable Pointcloud Implicit Surface Raytracing & Learned Properties Estimation (summary)

Summary video of completed thesis

More brushes



Editable pointcloud implicit surface: Sharpen, Tapered, & Uniform Retopology brushes

3 new edit brushes add new structured points & retopologize the existing pointcloud:

  • Sharpen: add creases like NURBS weight, except this allows a drawn curve on an arbitrary point surface. Existing features can be sharpened using other previous brushes.
  • Uniform: simple color line.
  • Tapered: brush width grows & tapers by a remap function, similar to Photoshop shape fade brush.

Added real-time smoothing of the drawing curve. Editing in a raytracing GPGPU pipeline is a bit odd, so each brush has specific requirements scattered throughout the raytracer. For example, the smoothing must be real-time to smooth each latest point because the neighbors disappear once a ray finishes.

Real-time Editable pointcloud implicit surface

Implicit surface updates to editing in realtime.
Edit functions: change radius for smoothness or sharpness, draw & visualize point attributes, draw new points along surface or extend to fill holes.
Octree leaves artifacts during editing; restores to normal after file save & reopen. It’s a tradeoff of speed for blockiness. I’m considering if brush effect can remain consistent across voxels without sharing data. Thread syncing is fast but threads/rays of different SM’s that hit adjacent voxels can’t sync.

Adding read/write ability to data on GPU does cost some overhead, along with extra management of new points and changing data. Fortunately some tweaks and cleanup sped up the original timing, such that the view updates in realtime while editing.

The new points appear after a drawn stroke since updating immediately allows point strokes to fly off the surface. It’d be good for drawing fur, like one of the tangent brushes in zbrush, but the purpose here is to add points on the implicit surface.

Two other methods to control feature sharpness’ll be shown later if I can get them to cooperate with the editable setup.

Houdini PLY Exporter

Houdini fileSOP uses gply external command to convert a [b]geo geometry to ply (in Houdini bin). Partly due to PLY’s flexibility, the output isn’t compatible with the PLY reader in Point Cloud Library (PCL), and possibly other common PLY readers. The PLY_exporter I wrote today is an alternative in a Python SOP node. The file header is in a simple heredoc variable in the script so users can easily customize it to exactly how their input software requires.

It’s bare-bones so it assumes P[xyz] and N[xyz] point attributes. Primitives should behave the same as gply. For my raytracer application, I transfer primitive normals to point normals, delete primitives (faces), then export the points & normals. Actually normals is not necessary since the app can estimate normals. In the figure notice most apps don’t use Pw (weight) for polygon/point data, but seems there’s no way to stop Houdini from outputting Pw.

(Left: PLY_exporter. Right: gply)

Simple GUI:

I’m not familiar how to share tools besides email. So the host could be slow:
Download PLY_exporter

Real-time implicit surface raytracing

xyzrgb dragon 1.2 million pointcloud

Count: 1.2 million points (1204929)
Model: XYZRGB dragon decimation (nonuniform point spacing)
Speed: similar to bunny model
Implicit surface & disk renders of pointcloud with normals, diffuse & shadow.

Octree construction memory decreased by 1/1600 to allow models greater than 400k points.

Normals estimation: 2 learning iterations
Normals unification: Multi-degree with metrics confidence, angle & residual.
Radius for shading: geometric pair fill & triangle fill



Reflection & Radius fix

Fixed major bugs in shading radius. Now speed is almost constant regardless of points count.
Fixed a reflection bug. Larger models are slower in reflection.
Models: 36k, 435k & 1204k points.
Varying radius affects both shading & reflection smoothness.


GTX 465 is quite old by now, but after the bug fixes the speed is around 30ms at the default far view, then gets slower with more zoom. Reflection’s about 4x slower, with 5 rays per pixel it seems reasonable.


Raytracing PCD as Implicit Surface

Implicit surface: higher quality than disk render, more accurate surface silhouettes.
Raytracing at interactive rates (video sped up)

3 clips:
1) No octree
2) In octree, radius 0.1: takes 60% time, very small speedup (compared to disk) since smooth surface requires more points. A denser model will benefit more from octree.
3) Radius 0.05: 36% time of 0.1 radius is a significant speedup. Smaller radius has less octree artifact due to less overlap, but less points mean it is statistically less stable as shown by the noise around ears.

Hierarchical LOD by Clustering & Frontface fix

Clustering – Point Cloud Simplification

Read the rest of this entry

Raytracing PCD as Implicit Surface

Iteratively project point x onto implicit surface, until x converges:

Read the rest of this entry

Diffuse 02 – Point Cloud Octree Raytracing

Video details: Diffuse pass, 1 directional light.
1) Octree partition artifact: overlap points duplicated across octree partitions to remove holes.
2) Redundant points from different leaves: fixed to avoid shading an overlap point twice.
3) Some backface and frontface fixes (a bit hacky), so the far frontface gets occluded by near frontface, regardless how large a leaf is.
4) Smooth Silhouette: removes the “disk” look.

Diffuse 01: previous artifacts


Octree Partition artifact:
Any duplicate points may cause discrete shading, which isn’t addressed in any papers that suggest redundant points in voxels. Tracing 2 rays 1 pixel apart, pixel 85 hits 4 leaves while pixel 84 hits 5 leaves. 2 overlap points appear in 3 leaves. Each point can appear in up to 7 leaves, and both rays hit 3 of the leaves.
Overlap points:
p2 [85, 256]:      dist prune IN. pos: 1.398461 1.886459 1.376074
p1 [85, 256]:      dist prune IN. pos: 1.395789 1.871096 1.369734

Pixel 85 gathers 2 duplicates per point, while pixel 84 gathers 3. So the former shades 17 points while later 19.

Problem 1: Different nnr_count (final shaded points count) results in different shading. I didn’t find a shading method that ignores duplicates (without explicit duplicates removal).
Problem 2: Overlap points are weighted multiple times if they exist in multiple leaves the ray hits.

I considered over three methods beforehand, but assumed the problem wouldn’t be visible and trusted the way mentioned in the papers.
Solutions: Either need to ensure overlap points are “unique” at octree construction phase. Or include the entire voxel which a point overlaps, forming a kind of nearest neighbor voxel structure.

High % of extra overlap points makes storage inefficient. Previous papers cite 3x-8x increase. Overlap points grow at more than 2x per level when the level scale is small, so need to detect when to terminate a branch. Yet point count has great effect on time, eg 150 points 160ms, 75 points 90ms. So need to balance between overlap redundancy vs point count per voxel.


Get every new post delivered to your Inbox.

Join 319 other followers