Picking bug

Feb 5, 2012 at 9:19 AM
Edited Feb 5, 2012 at 1:05 PM

Hi :) First of all - great job on your engine, awesome piece of code :)

On to subject - I would like to refer to the new feature you introduced in 0.261, namely picking. In docs you wrote (quote): "Caution: There are occasional picking bugs with JigLibX, where it will sometimes not return the first thing clicked on, but rather an entity that is behind what you've clicked on."

I think that's not 100% aqurate. The picking problem I run into happens only with heightmaps. When an object is on a heightmap (like a ball in your sample scene) and you try to click it you get the skin of the ground as a result. When I tried to find the intersection point between cursor-casted ray and any physics object on its path it became aparent, that the first object intersected was the ground and not the actual object I tried to click on. Furthermore, the intersection point was not on the surface of the ground as you can see it, but actually on one of the "bounding-box-like" thingies of the heightfield, visible after you enable DebugHeightfield.

That problem didn't occur when I replaced a heightmap with simple model of the ground with mesh-based physics. Every click did return expected results - objects that I tried to click on.

So my guess is the bug you refer to is not because JigLibX returns "object behind". It does return the first object, but unfortunately (in my case at least) that object has been selected not by (as one could expect) clicking on the surface but by picking-ray intersecting with invisible construct that it should not trigger intersection with.

I would like to know if anyone else can confirm my theory? Or am I wrong and the problem is somewhere else?

Edit: I have noticed that problem back in v 0.26, when I tried to implement picking by myself and I can see that same behaviour in 0.261, so I guess it wasn't just my bad since your implementation suffers the same error.

Coordinator
Feb 5, 2012 at 8:24 PM
This is exactly what I was seeing as well. The results are straight from the physics engine so this may be an issue with JigLibX. One way you could get around this is to use a collision predicate that only collides with terrain and another predicate that collides with everything but terrains, cast two rays, using the two predicates, and then take the closest result of the two.


On Feb 5, 2012, at 2:20 AM, "trashadlo" <notifications@codeplex.com> wrote:

From: trashadlo

Hi :) First of all - great job on your engine, awesome piece of code :)

On to subject - I would like to refer to the new feature you introduced in 0.261, namely picking. In docs you wrote (quote): "Caution: There are occasional picking bugs with JigLibX, where it will sometimes not return the first thing clicked on, but rather an entity that is behind what you've clicked on."

I think that's not 100% aqurate. The picking problem I run into happens only with heightmaps. When an object is on a heightmap (like a ball in your sample scene) and you try to click it you get the skin of the ground as a result. When I tried to find the intersection point between cursor-casted ray and any physics object on its path it became aparent, that the first object intersected was the ground and not the actual object I tried to click on. Furthermore, the intersection point was not on the ground you can see, but actually on the "bounding-box-like" thingy of the heightfield, visible after you enable DebugHeightfield.

That problem didn't occur when I replaced a heightmap with simple model of the ground with mesh-based physics. Every click did return expected results - objects that I tried to click on.

So my guess is the bug you refer to is not because JigLibX returns "object behind". It does return the first object, but unfortunately (in my case at least) that object has been selected not by (as one could expect) clicking on the surface but by picking-ray intersecting with invisible construct that it should not trigger intersection with.

I would like to know if anyone else can confirm my theory? Or am I wrong and the problem is somewhere else?

Feb 5, 2012 at 9:16 PM

Well yeah, I can do that - except for situation, when I want to click on ground itself, for example to give units order to move. Everything would be fine and dandy if the intersection point was on the surface of the ground, but it's not :/

One way to get around that issue is to have separate, invisible object used only to capture ground clicks. It would have to have mesh very similar in geometry to the rendered heightfield, but it doesn't need any texture, it will be invisible either way.

Another approach could be to use object as a ground, but that has obvious limitations. Actualy I use that approach right now, since I don't need very large terrain or one very complicated in geometry.

Coordinator
Feb 6, 2012 at 1:34 PM
Edited Feb 8, 2012 at 5:59 AM
It might be worth bringing it up on the JigLibX forums, although I don't think it is under development anymore. I can try and look at the JigLibX code to see if I can fix it. I will also do a test with their sample project to make sure this bug isn't in the QuickStart Engine itself. I don't have much time for the engine itself at the moment though, due to long work hours. This bug is the reason why v0.261 isn't current the recommended release.
Feb 6, 2012 at 6:53 PM
Edited Feb 6, 2012 at 8:10 PM

I saw your questions in http://jiglibx.codeplex.com/discussions/25929 from May 2008 and from January this year didn't get any answer. The answers they provided suggested, that ray (line segment in this case) SHOULD intersect with heightmap surface, but strangely id doesn't, at least not always. Did you get the same behaviour? Sometimes ray will intersect with ground surface, and sometimes with those boxes visible with DebugHeightfield? Or is it just me?

I attached three screen shots of what I'm getting and now I begin to think that my first assumption wasn't 100% correct. It seems like terrain physics doesn't "look" like rendered geometry, look here:

That yellow-black-stripped pole is my click marker. Green and red arrows touch with each other in a point, where I set that object's position to, it's that markers Zero point. I set marker's position to where interesction point is, as reported by JigLibX. As you can see, intersection was a bit higher that the ground itself. But on next images it gets funnier:

Above it's almost accurate, but only by accident ;]

And here it's almost completely hidden inside the ground.

It looks like Y component of the intersection point is always near 0. Did you get similar behaviour?

PS and off topic: can i change objects scale after it has been created and added to scene manager?

This fails - marker is created, but modyfication to scale does not change it's size:

MarkerPole = SceneManager.CreateAndAddEntityByTemplateID(19);
MarkerPole.Position = new Vector3(55.0f, 4.0f, -75.0f);
MarkerPole.Scale = 10.0f;

While this scales the marker as intended, but I can't resize it either later, for example after key press:

MarkerPole = new BaseEntity(this, new Vector3(55.0f, 4.0f, -75.0f), Matrix.Identity, 10.0f);
SceneManager.AddEntityByTemplateID(MarkerPole, 19);

Coordinator
Feb 7, 2012 at 2:38 AM
Edited Feb 8, 2012 at 6:00 AM
Ahh, the images say a lot. We currently have issues where objects sink slightly into the terrain, mostly in areas where the terrain is not horizontal. There is currently an open issue for that problem. After the terrain smoothing is done the height data is then used to create the physics, if you press C in the demo to enable physics viewing and then press V it should display terrain vertices that JigLibX is using. When I have done this everything appears normal, so I'm unable so far to determine why thing sink into non-horizontal terrain.
I'm not sure if this the issue affecting your stuff though. I have a method in the terrain component which you can use to find terrain height at given X,Z coordinates. I believe there is a game message you can use to access it, you could take the coordinates you get from picking and pass the X,Z coords to that method and it may give you a more accurate result.
Coordinator
Feb 7, 2012 at 2:58 AM
Oh, and changing scale of an entity that is already loaded is not fully supported, mostly because it would be inefficient (partially), altering the scale for rendering is efficient, but altering the scale of physics requires removing the body from the scene and replacing it with a new body. I'm not sure if the tender component is currently listening for changes in scale, or if it uses the current scale each frame. I'm on a phone at the moment so I don't have the code in front of me.
Feb 7, 2012 at 7:11 AM
Edited Feb 7, 2012 at 7:56 PM
LordIkon wrote:
Ahh, the images say a lot. We currently have issues where objects sink slightly into the terrain, mostly in areas where the terrain is not horizontal. There is currently an open issue for that problem. After the terrain smoothing is done the height data is then used to create the physics, if you press C in the demo to enable physics viewing and then press V it should display terrain vertices that JigLibX is using. When I have done this everything appears normal, so I'm unable so far to determine why thing sink into non-horizontal terrain.
I'm not sure if this the issue affecting your stuff though. I have a method in the terrain component which you can use to find terrain height at given X,Z coordinates. I believe there is a game message you can use to access it, you could take the coordinates you get from picking and pass the X,Z coords to that method and it may give you a more accurate result.

Hmm, interesting. That may be so, but it seems like it mostly affects rays. For example when you shoot a ball in your sample scene it then does sink a bit in the terrain, but other than that it moves "on" the surface. But when I cast a ray, only then the Y coord is messed up (almost allways 0 or close to 0).

As for the C and V - yeah, you can see the vertices, but I am almost certain, that in some places they are displayed lower than the terrain itself (I'm also refering to terrain after optimisation passes). I will check that out again tonight when I get back home, but I'm almost sure I did see vertices of the terrain physics sinking into rendered terrain surface.

Edit: Note to the paraghraph above - just now I have read [workitem:5175] and I get what you ment with C and V when reffered to terrain physics generated before optimisation. But I'm not sure if that explains Y being close to 0 when detecting clicks on the ground :)

And thanks for the tip on scalling - I will remember to scale only those things I can use without physics component. Hmm, no wait... You said render component may not be listening to scale changes - that would be true I think, because the marker I use has no physics and It did not scale. Bummer :) But it's not that important right now, I was just curious if I was doing something wrong :)

 


 

Another edit: I have investigated further and found out something. Let me show you on image below what I think is going on:

Sorry for quality, that was hand-drawn :] That black line is the heightmap, blue arrow is where I click with mouse. I cast a ray (pink) from camera (upper right, that green monstrosity) to the cursor and get intersection point. One could think its X and Z will be where the cursor is, but no - it turns out X,Z point to some other point intersecting the ray and not the visible terrain, nor the physics component of the terrain (checked with C and V) but something else, ilustrated above with the gray horizontal line. That something has skinID attached to terrain but when I used MsgGetTerrainHeight and fed it with those X and Z I got OutHeight that was correct for that point (and marker was placed on the right height) but unfortunately that's not the point I wanted to get in the first place :/ At least we know that acquiring height from heightmap for specified [X,Z] point is not bugged :D

Furthermore it looks like that this "something" that intersects ray and pretends to be the physics of the terrain is almost flat. When I clicked further away from the camera from very low altitude the intersection point (and thus the marker) was more and more displaced according to perspective (do I even make sense?). I can post fragments of the code I use to get that results, but I'm sure you know exactly how I cast a ray and get intersection point, right?

Damn, I just commided another wall of text, sorry :)

Coordinator
Feb 8, 2012 at 5:35 AM
Edited Feb 8, 2012 at 5:59 AM

Just found the problem. The normal is storing the position, and position is storing the normal. To fix this, open PhysicsInterface.cs, go to line 103, and change this:

out info.normal, out info.position

to this:

out info.position, out info.normal

Normals are always between -1 and 1 at the very most along any axis, which is why your position was always close to zero.

Thanks for all the info, it helped me determine exactly what to look for. That fixes the picking issues so I have released 0.262 as a stable build, rather than a beta with known issues. As your code may differ from the original engine it will probably be easier for you to just change that one line of code.

Good luck with your project, I would love to see some screenshots as it progresses.

Feb 8, 2012 at 7:12 AM
Edited Feb 8, 2012 at 1:58 PM

I'm glad I could be of some assistance :D I will try out your suggestions tonight.

It's hard to speak about my "project" yet - I'm rather goofing around with your engine right now, testing what I can do with it and learning how to actually use it and make it to do what I want to be done :] I'm also a single developer coding something that (hopefuly) will someday look like a game, although I haven't started to work on the actual game yet, I still have too much to learn about game programming to even dare to say "I work on a game" :) Anyway what I wanted to say is that any changes to code you release are not a big problem for me as I'm basicaly just modifying SampleScene right now and having a blast doing that :) Of course, having SVN would be sweet but I know that making public SVN while you are the only developer of the engine right now (I keep wonder why the hell all of them left?) would be too much of a hassle.

If you wouldn't mind I will stick around and poke you with more questions from time to time, just so you not get bored :P In the meantime I'm gonna tweak Nuclex a bit, I miss the ability to change font color at will. I have done that when I first tested that framework, now I'll just replace dlls you included to QSEngine with my modified version of Nuclex's source.

Coordinator
Feb 8, 2012 at 5:17 PM
Yea, they left because they got jobs that paid money. I'm in a similar boat, but rather than quitting entirely I just work on it occasionally when I have the time. I figure that at the least it is good practice, something I can put on the resume, and if I can help others out at the same time then that's icing on the cake.

On Feb 8, 2012, at 12:12 AM, "trashadlo" <notifications@codeplex.com> wrote:

From: trashadlo

I'm glad I could be of some assistance :D I will try out your suggestions tonight.

It's hard to speak about my "project" yet - I'm rather goofing around with your engine right now, testing what I can do with it and learning how to actually use it and make it to do what I want to be done :] I'm also a single developer coding something that (hopefuly) will someday look like a game, although I haven't started to work on the actual game yet, I still have too much to learn about game programming to even dare to say "I work on a game" :) Anyway what I wanted to say is that any changes to code you release are not a big problem for me as I'm basicaly just modifying SampleScene right now and having a blast doing that :) Of course, having SVN would be sweet but I know that making public SVN while you are the only developer of the engine right now (I keep wonder why the hell all of them left?) would be too much hassle.

If you wouldn't mind I will stick around and poke you with more questions from time to time, just so you not get bored :P

Feb 8, 2012 at 7:17 PM

Hate to say it, but the bug still persists :(

I have just downloaded 0.262, verified in PhysicsInterface.cs that those two parameters are in the the right order, and they are, then I add my marker, set it's poosition to the intersection point and... bummer :/

Here's what I do, step by step, maybe I'm doing it wrong?

1. define public global object for SampleScene:

public BaseEntity MarkerPole;

2. In SampleScene's constructor I create my marker and add it to scene manager:

MarkerPole = new BaseEntity(this.game, new Vector3(900.0f, 204.0f, 500.0f), Matrix.Identity, 10.0f);
this.game.SceneManager.AddEntityByTemplateID(MarkerPole, 19);

3. In RayCastForEntityReport I do this:

MarkerPole.Position = info.position;
//just to get the point displayed
window.Label.Items.Add("SegmentIntersectInfo.position:");
window.Label.Items.Add("   X=" + info.position.X + " Y=" + info.position.Y + " Z=" + info.position.Z);

Coordinator
Feb 8, 2012 at 7:28 PM

I ran some tests using the RayCastForEntityReport and I was getting positions sent back to me that represented the point I was clicking, whereas previously I was getting something near 0.0 on the Y axis. What types of position values are you getting?

Feb 8, 2012 at 7:33 PM

Sorry, pushed "Save" too early...

Well, after i add all that to SampleScene when I click on the ground my marker is not placed where you would expect it to be - he's close, sometimes higher, sometimes lower and sometimes to the sides from the point of clicking :/ It's almost like the physics angine does not return correct point of intersection with heightmap.

And just now i realized why your change to the PhysicsInterface didn't change anything for me - I didn't use your RayCastForEntityReport method before as I created my own back in 0.26 that looked like this (I did it in QuickStartSampleGame.cs):

CollisionSkin skin;
Vector3 pos, normal;
MsgGetRenderEntity camMsg = ObjectPool.Aquire();
this.SendInterfaceMessage(camMsg, InterfaceType.Camera);

// Make sure the camera's unique ID is valid
if (camMsg.EntityID != QSGame.UniqueIDEmpty)
{
	// Now that we have our camera, grab it's forward vector
	MsgGetVectorForward getVectMsg = ObjectPool.Aquire();
	getVectMsg.UniqueTarget = camMsg.EntityID;
	this.SendMessage(getVectMsg);

	MsgGetPosition getPosMsg = ObjectPool.Aquire();
	getPosMsg.UniqueTarget = camMsg.EntityID;
	this.SendMessage(getPosMsg);

	pos = getPosMsg.Position;
	normal = getVectMsg.Forward;

	MsgCameraGetValues camv = ObjectPool.Aquire();
	this.SendMessage(camv);

	view = camv.ViewMatrix;
	proj = Matrix.CreatePerspectiveFieldOfView(MathHelper.ToRadians(camv.FOV), camv.AspectRatio, camv.NearPlane, camv.FarPlane);
	Vector3 nearsource = new Vector3((float)mouseX, (float)mouseY, 0f);
	Vector3 farsource = new Vector3((float)mouseX, (float)mouseY, 1f);

	Matrix world = Matrix.CreateTranslation(0, 0, 0);
	
	Vector3 nearPoint = GraphicsDevice.Viewport.Unproject(nearsource, proj, view, world);

	Vector3 farPoint = GraphicsDevice.Viewport.Unproject(farsource, proj, view, world);
	SimpleSkinPredicate pred = new SimpleSkinPredicate();
	Vector3 direction = farPoint - nearPoint;
	direction.Normalize();
	//Segment seg = new Segment(ray, Vector3.Down * 1000000000.0f);

	Segment seg = new Segment(nearPoint, direction * 1000000000.0f);

	CollisionSystem cs = PhysicsSystem.CurrentPhysicsSystem.CollisionSystem;

	cs.UseSweepTests = false;

	cs.SegmentIntersect(out dist, out skin, out pos, out normal, seg, pred);
	if(skin != null)
	{
		//Collision
		skinid = skin.ToString();
		QuickStart.Physics.JigLibX.QSCollisionSkin s = (QuickStart.Physics.JigLibX.QSCollisionSkin)skin;
		selectedEntity = SceneManager.Entities[s.EntityID];
	}
	MarkerPole.Position = pos;
}

skin predicate:

            if (skin0.Owner != null)
                return true;

            else
                return false;

I may have forgotten to copy some definitions here, but all in all the code works great, it just doesn't place the marker on the right spot :/

Any ideas?

Feb 8, 2012 at 7:36 PM
Edited Feb 8, 2012 at 7:37 PM
lordikon wrote:

I ran some tests using the RayCastForEntityReport and I was getting positions sent back to me that represented the point I was clicking, whereas previously I was getting something near 0.0 on the Y axis. What types of position values are you getting?

I'm getting almost resonable coords, X and Z are "good", Y is around 50-200 depending on the point i click on but... Even if they are almost good they are not true "click coords" :/ Try to position some object (tall object) on that coords and see for yourself :)

Feb 8, 2012 at 8:31 PM

I was able to get a screen recording of my clicking, here it is:

http://www.youtube.com/watch?v=PYDhWyk00Uw&feature=youtu.be

You can see where I click and where the marker is being positioned. I hope that can clarify something :)

Coordinator
Feb 8, 2012 at 8:46 PM
trashadlo wrote:

I was able to get a screen recording of my clicking, here it is:

http://www.youtube.com/watch?v=PYDhWyk00Uw&feature=youtu.be

You can see where I click and where the marker is being positioned. I hope that can clarify something :)

That definitely seems to be buggy. I can try and take another look at this when I get a chance, that might be a few days however. I'm hoping it's a problem with QS and not JigLibX, as I really don't want to dig into their code and try and fix this if it is their problem.

Feb 8, 2012 at 9:39 PM

I'm affraid it might be physics fault since I got buggy results both with your ray collision detection and my own, posted above. I will try to see where that error might come from, but I'm not that smart to dig into physics implementation :) Nor the shader implementation, I'm sure that IS black magic barely disguised as some kind of mystic code-like hieroglyphs :)

Luckly that picking bug is not holding me back on anything, as I said waaaay back in my second post for the time this bug is present I use simple object based ground and focus on implementing point-and-click-to-move behaviour. Then I'll think on pathfinding :)

Feb 12, 2012 at 7:47 PM

Hey, me again :)

I may have said that "this picking bug is not that much affecting me" a bit too soon :] I have played a bit more with clicking on heightmaps and clicking on objects pretending to be ground and I found out another anomaly, possibly related to passing mesh to physics engine. Look at this video (this time with my annotations): http://www.youtube.com/watch?v=lfvgLQA7cwg

It seems like that ground-like object (made from triangles only) when passed to physics as TriangleMesh is getting messed up somehow, like vertex indices are combined to create triangles that are not in the original mesh. I belive this creates more triangles that are necessary and (what is more important) that invalidate the physical skin. If you position the camera on another angle, as I did in the above video, you can click "below" those triangles and get correct intersection points with object's surface. When I enabled PhysicsRenderer in that movie you can see edges of those mysterious triangles.

Does that assumption make any sense to you? Do you think that may actualy happen and possibly be the reason for messed up heightmap clicking also?

Another thing I faced was that in some cases the ground object disappeared (you can see that at the end of the video), but I think i'll write about that in another thread, so this one is for the picking bug alone :)

Coordinator
Feb 13, 2012 at 12:30 AM
Edited Feb 13, 2012 at 3:04 PM
Yea I think I've seen issues with triangle mesh shapes before. I may be able to look into this more this week. I'm curious if there is any mention of triangle mesh shape issues on the JigLibX forums.
If you look at the sample demo that comes with the engine, the ship model's physics have some minor issues where a few vertices are not where they should be, and that is also using triangle mesh shape physics.
I would caution you against using a triangle mesh shape for terrain, I haven't seen the best performance with them, and have seen issues like things falling through them as well. A heightfield terrain has performance advantages due to the fact it cannot have more than one point at any given X,Z coordinate. The main case for using triangle mesh shapes for sections of terrain is so you can do formations of terrain that overlap, like a land bridge, or an arch, or caverns, etc.
EDIT: Just glanced over the forums at CodePlex and on their wiki, and found nothing about issues with ray casts or triangle mesh shapes. They haven't patched since Dec. 2010 so I'm assuming there will be no help with questions on their engine any longer. It's somewhat frustrating that there has yet to be a well supported physics engine for XNA after 5 years. I had thought about making the engine PC-only at one point and just using a wrapper over a C++ physics engine, however I would prefer not to limit this engine to PC. I doubt I will be able to fix the problem with triangle mesh shape generation, but the picking issue I will look into some more. In the video you just posted the picking seems to be working perfectly, it's just the triangle mesh shape is malformed and causing issues. That shouldn't be the case with the heightfield though, so I'll look into it. I'm hoping to get a chance to look at this in the next couple of days, but no guarantees as I work about 55 hours per week, and spend another 10-15 hours per week commuting.
Coordinator
Feb 15, 2012 at 5:49 AM
Edited Feb 15, 2012 at 5:50 AM

I've reproduced the picking bug against heightfield, unfortunately it seems to be happening all over the place, mostly on slopes. Even worse, it seems to be a problem from within JigLibX itself. There are places where a raycast passes right through terrain and goes another few hundred units, but physics meshes bounce right off that same location just fine. Rendering the physics heightfield also shows that the vertices of the physics mesh are where they should be.

My guess is that the problem lies somewhere in SegmentIntersect() within CollisionSkin.cs. The full path is JigLibX project->Collision/CollisionSkin.cs.

Currently looking at this:

http://jiglibx.wikidot.com/forum/t-140261/terrain-ray-casting

Feb 15, 2012 at 6:33 AM

It may be related to that bug report:

http://jiglibx.wikidot.com/forum/t-70791/heightmap-ray-test-bugs

Coordinator
Feb 15, 2012 at 4:19 PM
trashadlo wrote:

It may be related to that bug report:

http://jiglibx.wikidot.com/forum/t-70791/heightmap-ray-test-bugs

Possibly, although that bug is a year older than the newest JigLibX build, so I would hope they had fixed it by then.