Knee pop… damper on n stretchy joints
(notes too long, attempt at coherence…)
Placing the knee damper into Andrew’s rig is a challenge. It has multiple bone scale controls, and a double knee joints setup.
The original script should work with controlled scaling, but I need to see where to insert the script so there won’t be unstable (cycling or conflicting) scale operations.
Investigate leg rig:
Stretchy_Limbs expression scales bones to reach overextended IK controllers.
This effect can be on/off.
Distance node (R_hip, R_anke) shows right leg longer than left.
The constant value 36.700 in DivScaleLimbs is the total length of all leg bones, confirmed by checking the thigh+shin+small_knee to be 36.7031111. So stretching should only occur when all knee angles are pi.
Full rig scale also affects the distance node. I suppose it corrects it for global scale difference. I’ll ignore this for now.
Fixed: frames where the controller’s exactly at full exstension, remains a scale=1.0.
/*expression error example
An expression linked channel is purple only when the expression has a direct assignment of form “obj.attr=value”. setAttr will not create expression link.
Linked channels update correctly up to 1 nesting (ie assignment is in a top scope like if, for clauses). 2 or more scope nesting may produce unstable results (eg if{ if{} }). One situation will have about 3 separate answers per frame depending on timescolling, step forward to frame, step backward to frame, & jump to frame.
Any non expression commands (like tag based, eval “ etc) makes the link unstable.
*/
if(joint25.rotateX<1){
if(time>1){
joint26.scaleX=2;
joint25.scaleX=2;
}
else{
joint26.scaleX=time*2;
joint25.scaleX=time*2;
}
}
Answer: I think this is undocumented in mayadocs? So I can destroy all expression links and make them purely “script-like”. Or limit the scope nesting. Placing non exp commands into a separate function gave the same unstable results. Not all the knee pop commands can convert to exp friendly, so I need to “scriptify” the whole expression.
Now it is STABLE! Damping works.
However, damper works under a particular scope of the scaling expressions, so when the leg transitions out of the damper scope, it is NOT smooth. Due to the leg still bent, so damper needs to stay active until knee is completely straight.
This also means both stretching needs a specific threshold length, and the damper needs to use a particular distance measure. It’s trivial and working when a leg has 2 bones, but a double knee changes distance assumptions.
Generalizations: IK chain, a start and end bone, n bones in between approximate an arc. OR, majority of leg length is start/end bones, and n very small bones in btw approximate 1 knee joint.
Error: bad approximation where answer either doesn’t converge, or converging from the wrong side of a limit.
Error 1: Theta extracts an offcenter angle, so:
smaller theta (than true knee theta) => smaller x2
Fix: No need for virtual tri. Just angle of the virtual knee.
Error 2: smaller leg length => smaller x2
Fix: need to assume thigh, shin form equilateral triangle. Obtuse triangles (with partially unknown lengths) require more calculation.
(…blah blah TMI…assume full equilateral symmetry…)
Ratio: 0.963 //using $lThigh as triangle side
Ratio is constant when correcting (using original theta1), suggests equation is correct to a constant error.
Ratio: 1.041 //using $lShin as triangle side
Thigh and shin lengths different: 17.22805234, 18.66578588
Use avg length helps approx.
Ratio: 1.0 //WOW! Equations exactly match => approximation correctness.
Once theta2, the corrective angle, is applied, it all works smoothly. WOOT!!!
Video of leg:
stretch: original stretchy to reach overextended IK control
damp: this script
damp+stretch: the 2 working happily
Reviews: AENY, Autodesk demo
wow, hard to find time to write up these events I went to… Went to some showcases of new/up-coming software.
AENY: Aftereffects group in NY showed AE CS4.
So basing off of presentation only, my thoughts on the features…
Mostly it’s the bridge improvements btw the Adobe products, autoupdate btw softwares without exporting/converting/rendering. So I’m glad AE is super close to PremierePro now. Flash came from a different company’s code, but they managed to have an export function from AE to Flash (CS4?). The amazing part is all animated elements are separate and layered in Flash, and when he scroll the timeline — all animation clips actually played!!! Amazing!! In my Flash (CS1 or 2?), I can’t see anything on any clip/object/group except the first frame, it was the most retarded issue I found that made Flash not fitting for character/scene animation. It’s still far away from ‘real’ 2D animation programs (and lightyears away from anim apps from Japan…).
I only recall 2 new filters he demoed. Talking about cartoon, there’s a toon filter, complete with ink and paint. The Ink looks quite cool, some tapered lines, naturally broken, very (I mean VERY) consistent across frames (from a live action noir film). Paint has the usual simplified grades of shades.
Bilateral blur? He could selectively blur an image based on detail, so areas of low detail/light becomes more blurred, retaining the sharpness of “important” features. It looked like a simple set of Matlab instructions, just take a mask from a high pass filter, then convolve blur based on this selective mask.
Speech recognition into words I’m not too interested. It’s just a convenience by dragging in a research paper (well, implementing well is another art). You can drag the text to the timeline, and markers will show where each word is spoken. This is handy especially with the now ubiquitous search tool (search any project files, layer properties, filters etc).
AE also added a 2.5D plane based motion tracker, Mocca? It’s standalone, but exports quickly to AE. It’s $300 on its own, but let’s see if this removes tracking headaches?
Autodesk demo: last week? It was kinda small and unorganized (let’s hope the 3December event on 12/2 would be much better).
Motionbuilder 2009: Mostly general base structure things, like good hardware shaders in view. The Python is improved? If it’s similar to Maya’s then I might try it. Long over due: 64bit and Vista OS’s support (when Win7 is almost here to save the day?). Other Autodesk software have worked on these platforms long ago. This $4k mass is a bit slow to move, but it’s good. The new dynamics looks like a port of Maya’s realtime dynamics (ages ago), and 3dsmax’s ragdoll simulation (four versions ago).
Edit: must’ve forgotten this cool thing. mb’s FBX format can be viewed in quicktime standalone, great for presentations. If it works with any browser without plugin download, then this can be the winner to the decade old web 3d problem. Currently most web 3d either requires a plugin download (viewpoint), uses java applet (like my 3d projects for Ken Perlin), or a custom server backend streaming to client flash page (also no installation, like sosauce and most web companies).
Mudbox 2009: Essentially many photoshop brushes in 3D. Also hw shaders, which when applied does look amazing, realtime lighting, nice shadow, bake in ambient occlusion, all in high poly count. They mentioned the workflow can subdivide up up 45mil polys. Nice. Clean interface. But I wonder if it can compete with ZBrush?
Maya 2009: Lots of focus on workflow efficiency. So most enhancements already had well established scripts online (like interparticle collision or proxy).
Soft mod is also placed as a type of (soft) selection, exactly the same as what 3dsmax users are used to. This makes it much faster for modeling, instead of deformer setups, then delete history…repeat…. The presenter didn’t show the muscle feature (from 2008 paid ext2), like to see how stable it is. nParticles looks awesome, lots of sticky, waterish possibilities–dripping sweat along UV coords used to be a common mel headache for many of us, but now it’s built in…drip, stick, flow, linger, accumulate, drip~0~0~O. A grid of particles can act like cloth sim, so it seems to replace classic particles’ ability for softbodies. On a dual core laptop, it’s realtime playback is actually fast enough, where he could interactively move a bowl and splash water around (one of the main concerns, cuz default ncloth is awefull slowwww!!), I’m assuming the 8GB ram didn’t play a roll! I doubt any script could achieve this speed, maybe a proprietary C plugin. However I’d like to see exactly how parameters are layed out and controlled, if scripts really become unnecessary. I see those ramps typically in shaders/paintfx appear in the UI, to control speed, stickiness etc, but if harry rotations, material assignments, and event triggers are streamlined, that’ll be awesome. Btw, nparticles can be converted to polygons (waay overdue), so no more buggy script converters.
UV editor and render passes both are streamlined very well, interactive uv relaxing, less trial/error, more tweaking using sliders. Massive render passes is easier to manage/create, and mentalray seems to optimize pass rendering per frame, so instead of 21 separate renders (in 2008) per frame, mentalray renders once, and all those passes layer up. Subjectively it looks like 5-10x faster (i duno cuz he didn’t compare the rendering in 2008).
Stereo 3D camera, with several viewport styles, like the classy red/green images. Probably made for Dreamworks folks, and others. Hope to see yall in trendy cardboard red and green sunglasses!
Animation layers. I think it’s very cool, mainly easy to use. But most people probably are used to layering animation using trax editor character clips, multiple weighted constraints, deformers, parent/groups (old school but tried and true). The layers of keys are additive and can have weights (same as trax right?), but they actually are color coded, awesome! See blocking keys in color x, emotions in y, details in z etc. Dreamworks a long time ago (4 yrs?) had a script to color code keys in the timeline into keyframes and breakdown/inbetweens.
mocap: directshow-optical flow-vicon server-motionbuilder
Starting out with a lot of components for this project. I spent quite some time getting Directshow to work with OpenCV. There was a library for “easy” Directshow use (not specific to opencv), but the directions were ambiguous, mostly outdated, even though the library itself is fairly updated. This weekend got Visual Studio working (previously it would die mid install), opencv and directshow now happy, which means my cellphone can disguise as a motion tracking web cam. Here’s an idea: let everyone stream their cellphone camera video onto a web database 24/7 (u can call while videoing), this’ll reconstruct an entire 3d textured world (not quite dark knight batman style), lol the data would be huge!!
Later I’d need to connect opencv to motionbuilder using our classmate’s server. It’d be pretty cool realtime, but I’m feeling lazy.
Also need to read on research papers about motion analysis. This markov random modelzz might be useful….?
btw I also fancy linking matlab code to opencv somehow, that’d be fun…wait, if I just use matlab what’s the point of a buggy, backward styled, mostly non OOP opencv? I dunno *arms waving for a good excuse*
EDIT:
Each part works separately, then yesterday unionized all of them. In total a few libraries and connections.
supported camera/cellphone -> Directshow stream input/filters -> opencv library -> detect simple average motion with optical flow (fast Lucas Kanade algorithm) -> vserver output (thanks Nadim!) -> vicon RT plugin in motionbuilder -> a realtime moving marker!!
Roger’s gona build that nice 3D motion detector, so tho the current avg motion gives weird numbers sometimes (jumps to far left), I’ll just wait for his stuff.
Seb’s doing whatever’s possible to have a dancer in motionbuilder react by parameters.
I still need to figure out this mb 7.5 ext2, it’s unsupported on vista, so I get buggy behavior, especially the dreaded 50% cpu idle. mb 2009 officially supports up to vista 64 bit, so I’m looking to try it out. Hopefully then my fps will rise.
K means, EM, structure from motion
a crazy weekend today, vision project took a day, for extracting camera parameters from 3d points matches with their 2D projections on the camera image, and a cooler process of using 10 frames of matched points to extract both camera parameters and estimate the 3d point locations from the 10 2D images.
It’s pretty cool, but only the core. There are lots of other problems in practical use, including camera calibration (distortion, amorphic etc), and how to create point correspondence.
Finally, we reach unsupervised learning in our ‘machine’ class. I think it’s a class of algorithms more practical, but maybe less accurate? Unless there’s an automated way to collect databases of output labels, unsupervised should take the bulk of processing. So 2.5 days of K means clustering, and expectation maximization algorithm (EM) using gaussian mixtures. The obvious limitation is K is manually set, but there are methods on efficiently searching the optimal K prototypes in a 0…Infinity space, though I doubt that maximization can reach too high a number. It will become like finding the K best features, and ignore the rest that might be noise.
Machine learning – neural network and modular learning
This back propagation forward progagation business, even though was in serial modules, remained harry. Only 3 days of translation, half the time of the previous homework, but the later coding/debugging took awhile, prob cuz other classes got buzy too.
It’s just like a neural network: each module (node) inputs data, performs some simple tasks. But collectively the purpose and process seems very mysterious.
Challenges include understanding the purpose of propagating. Now I found it involves ‘filtering’ the input sample, weight vector, and weight vector gradient. Other challenges…reasoning between many functions/classes/variables, since it’s far from a linear command program, often tracing a variable’s lifetime change through multiple modules is same as fixing pipes, or intestines, from the inside. Some of the translated code template is never used, which confused me some time.
OOP can minimize code while forming process complexity. Naturally some processes, like altering and outputing variables, is useless. An object function is designed to output in two directions (like a train), but the unidirectional head and tail trains will always do wasted extra work. ==> good case of code readability vs efficiency. But readable doesn’t mean ‘easy to debug’!!
Any numerical input database works on this neural net (I don’t know which type is better suited tho…). Here professor gave the UCI isolet database, data describing single voiced alphabets. I’m quite amazed my best simulation score got 3.3% error (96.7% accuracy!). So it’s great for voice recognition. But real life has connected syllabus chains and semantic context dependent, so that’s another thing. Unless a preprocessor breaks an audio file into alphabet pieces, then neuralnet can aid in a larger context based algorithm.
Knee pop… minor but obvious
(EDIT: educational video added)
IK Leg: Knee Pop
Knee pop: the large angular change when an IK chain reaches full extension, and bones appear to lock to the extreme for some frames.
For a standard 3 joints (2 bones) IK chain, high angular velocity near full extension is no surprise, appearing to ‘pop’. In fact, a pop should occur.
Briefly explaining: the knee angle only depends on the distance of ankle to hip joint. The IK system inputs a translation from IK control, outputs angles to have the ankle match the IK control location. On a unit circle, a conversion example:
theta=acos(x/1)
The graph of above equation, with full extension (-1) or minimal extension (1), the derivative approaches infinity (vertical). It’s obvious when you see the 1st derivative:
d_theta=-1/(1-x^2)^0.5
Solutions?
Most animators simply take care of the keys and curves to avoid full/hyper extension, since those poses tend to be rare. Professionals on the pop:
http://animationtipsandtricks.com/2008/08/readers-question-workflow-and-timing.html
In Maya solutions:
Angle limit: works, but animators want to reach full extension. Also, often limits cause weird jitters and IK could start twisting along x (bone axis).
Angle damp: works on 4+ joints, so not for 3 joint legs.
Autocorrect: fix bad IK control positions by adding ease-in and limiting distance to hip. Need to figure the ease-in amount.
Looking at the derivative, simply counteract the angle speedup by setting IK control translate keys with extreme ease-in eas-out. You can calculate the best ease, but most animators prefer to determine that speed right?
Compression of Motion Capture Databases (UTA, Siggraph 2006)
This paper mentions it’s common in commercial apps to allow imperceptible bone length changes to minimize knee pop. Perhaps it’s hidden in 3d apps or motionbuilder has it, I’ve never heard of this. It’s compression results are amazing, but here we’re dealing with a simple problem. But we seek the easiest solution that maintains animator useability.
Footskate Cleanup for Motion Capture Editing (UW, Siggraph 2002)
This paper implements several methods on mocap cleanup, one of which fixes knee pop. I didn’t read the full paper, so seems like the corrective component of the system can’t be used separately. Are they proposing 2 methods? Either knee damping, or leg stretch methods?
Given prebuilt IK solvers (like Maya), damping the angle is unlikely, since it’s part of the solver. I can write a plugin solver with ankle damping…
Leg stretch is nonrealistic, but common.
Further reading shows they damp the knee after IK solve, and stretch the thigh and shin bones to reach d (distance btw hip and ankle). It’s not mentioned, but I’m assuming they rotate the hip afterwards to match the ankle to IK control exactly.
Kind of envy how they are in control of the whole system. In maya it’s better to only adjust the IK control or distance btw things. IK’s supposed to deal with all angles.
…just got a working expression. First one cut off 10fps, a simple cleanup made >1fps hardly any slowdown. It uses several cosines, so I’m considering a pseudo-linearish solution if performance matters. I’m quite amazed by the smoothness, by indirectly damping the angle. It doesn’t have the exact equations from the papers though. I opted for a cosine damper. Later I’ll think of a simpler/more user directable damper. That’ll be useless for small joint numbers where animators just fix it, but assuming a large IK system (whatever that is???), it could help dampen annoying pops.
PSPad rocks!
few programs make you excited about plain text. But learning this, it’s quite filled with useful functions (reeely useful). No wonder my Maya mentor Pzman uses pspad, so much that he defined the entire library of Maya command/syntax in pspad, and created a pipeline bridging pspad to maya execution.
I don’t have the bridge. But if I need it, there are several made by a few pros online. Or I can just ask him.
machine learning: prop & gradient based learning
Starting I think friday? Maybe I should do both projects simultaneously.
Hoping translation from lush to matlab is 2-3 days. Then implementing the modules and running isolet database would be the main bulk.
When he talked about bprop and fprop, I had no idea how the neural network thing worked. I still don’t, so I gota review the material. Thankfully the base code has lots of explanation this time.
…looks like I need to improve on everything, but at least i wont fail the class soon, since failing a grad class looks aweful to any prospective grad studies applicant.
computer vision
trying upload hw1_img_features_small
computer vision, mocap
Whoo! This week off loaded two projects one homework. I’m glad the programming thoery hw WAS as short as a hw. The others are really projects.
mocap we did openCV, improving a slow face detection using sparse optical flow of tracker points. Mine worked, by specification, but seems like the face detector itself got slower by 50% (not counting the flow code), without any reason. It’s the same video file, so…. maybe i altered the video, will check later,
I finally got a webcam to work on my laptop, capturable via Virtualdub. Unfortunately opencv doesn’t see it. It’s weirdly setup, cuz the camera is embedded in my cellphone, so the video goes thru a funny app I installed in windows mobile phone, thru usb, and into a laptop. It does work with Virtualdub, Skype, and probably most ‘modern’ apps.
A directDraw thingy? maybe, it might work out in opencv, since it’s all C/C++ anyways.
Computer vision, fninished implementing the Harris Corner detector, using Gaussian, or was it the delta matrix? can’t remember. It’s ok algorithm, some points don’t lie exactly on corners, maybe near. Some point, perhaps due threholding, see to lie on an edge, but to humans it doesn’t look to be a corner.
Then fun with scale invariance, help from Laplacian filters, varying either image size or sigma size of the filter. Recursively halving the image to use the filter to detect blobs is pretty fast, but at some point I think the scaling or sigma needs to change in lesser increments than 2^n. When the filter convolves with the image, we get a ‘response intensity’, of how an area sigma sized might be ‘a’ blob. So the size of blob is dependent on sigma size and image scale factor, unless other methods are used to extract the size (clustering, aggregating, anyone??).
Lastly image to image alignment took the most time. All three of these algorithms were done in under two days, but some errors in image alignment dragged on. It uses SIFT descriptors from a prebuilt program (thanks to David Lowe), it’s awesome for describing ‘unique’ feature points. Then run iterative RANSAC to match points, compute potential transformation, and transform image 1 into the affine transform closest to an image 2 appearance. After fixing a stupid conceptual error I had on SIFT, the algorithm worked like AWESOME, under only 20 RANSAC iterations. It’s probably more accurate than even manually photoshoping it!
