BE - Spaces in path

Dec 16, 2007 at 12:04 AM
This is a discussion for BE, spaces in path

It is correct that spaces in the path is a problem, it's not a issue in BE per se, but in the command processor for DOS. There is no fix for that.

There is a workaround and that would be to use subst during install. This way a new virtual drive would be created for the environment and you would simply use that. I.e. it would set up the X: drive to be the source root, so instead of "c:\my documents\visual studio 2005\projects\quick start engine" you would just have "x:\"

Would this be an acceptable solution?
Dec 16, 2007 at 12:10 AM
I have spaces in mine and it still finds it and works...well the gay merging dosn't work, and now that I think of it, that could be the problem, but the thing still runs and does everything except merge.

As to your other solution, I like that to. Seems simpler and I think it would be much easier to keep track of my project like that.
Dec 16, 2007 at 12:15 AM
That's fine, as long as its configurable.
Dec 16, 2007 at 12:18 AM
Not to hijack. How do I officially join this project? Do one of you have to add me or is there somewhere that I have to join at?
Dec 16, 2007 at 1:04 AM
Done.
Dec 16, 2007 at 3:31 AM
Thank you, I now return you to your previously scheduled thread. Thank you for flying hijack air(what a horrible name for an airline)
Coordinator
Dec 16, 2007 at 3:43 AM
Yea, I think I'd rather drive instead.... :oP
Dec 16, 2007 at 1:06 PM
Edited Dec 16, 2007 at 1:06 PM

shawmishrak wrote:
That's fine, as long as its configurable.

What should be configurable?
  1. Possible to set the drive letter
  2. Possible to choose if you want to subst the path
  3. All of the above
  4. None of the above
Dec 16, 2007 at 5:30 PM
Let the drive letter be configurable.
Dec 16, 2007 at 7:16 PM
I no its a kludge, but why not use QuickStartEngine as the folder? I am as it matches the cpc project name. You just cant guarentee how many drive letters we have available. (And I dont like messing with pc settings for each of my projects)
Dec 16, 2007 at 8:24 PM
The issue is there can be no spaces in any of the path. So if you're on XP and put the folder on your desktop or My Documents folder, you're screwed. It will be placed at C:\Documents and Settings\User\Desktop, which contains spaces.
Dec 21, 2007 at 9:42 AM
As it turns out cpc can't handle projects being at the root (as it will when you use subst) I made a custom build fixing that and will apply it on the next submit. But there is still an issue which I like discussed.

When you checkout the code the current checkout is bound to the path, this means that you have to subst before you checkout. I have no problem with that, since you have to create a folder and run cpc any way. I can create a batch file which you can run from the desktop and prompts the user for a drive letter and then does the checkout and runs setup so everything is ready.

What do you think?
Dec 21, 2007 at 11:07 AM
Got two major problems with successfully running the project:

  • Downloaded cpc yesterday.
  • Created a directory on my 'd' partition:
d:\Programming\qsengine\
  • Switched to this directory.
  • Downloaded the current source:
cpc checkout QuickStartEngine
  • Updated the source (hope this was the right way to do it):
cpc update

But I don't think, 'update' did anything at all.
Now how exactly do I apply the patches found on a tab of the source code site of the project?

Furthermore there were 2 problems using the 'unpatched' solutions:
At first I had to make the 'game' projects of each solution the active startproject for the compiler to begin - not a big deal.

QuickStart_Windows.sln - works, but I get only a black & white screen where white must be some kind of terrain.
No texturing, no lighting etc.
relevant computer specs:
NVIDIA GeForce 7900 GT/GTO
AMD Athlon64 X2 4800+
2 GB of RAM

Prototype.sln - ComicSansMS missing... somebody mentioned to just replace the font. But there was no 'Content' folder applied to
the project. Well, at least not visible within VS Studio 2005 Academic Edition. Didn't try to open the solutions with Express, though.

Any suggestions, hints?
Dec 21, 2007 at 11:32 AM

Suzume wrote:
But I don't think, 'update' did anything at all.
Now how exactly do I apply the patches found on a tab of the source code site of the project?


Corect update doesn't do anything if you just checked out the newest source (You did by running checkout)
In order to apply a patch (Don't do that in a folder you are working) download the patch and then run "cpc applypatch patchname.extension"
Unfortunately there is no nice way of removing a patch, you have to do that manually.


Suzume wrote:
Furthermore there were 2 problems using the 'unpatched' solutions:
At first I had to make the 'game' projects of each solution the active startproject for the compiler to begin - not a big deal.


I thought I had fixed that, oh well maybe in a later checkin
Dec 21, 2007 at 3:52 PM

Sturm wrote:


Suzume wrote:
Furthermore there were 2 problems using the 'unpatched' solutions:
At first I had to make the 'game' projects of each solution the active startproject for the compiler to begin - not a big deal.


I thought I had fixed that, oh well maybe in a later checkin


I'm pretty sure that can't be fixed without saving the VS user options in the source tree, which we don't want to do. It's just a quirk of VS.
Dec 23, 2007 at 4:59 PM
Edited Dec 23, 2007 at 5:06 PM
Reproduced the first 'error' on another system.
Black sky, white terrain.

System Specs:
NVIDIA GeForce 8800 GTX
AMD Athlon64 X2 6000+
2 GB of RAM
Both systems got a running version of XNA 2.0 and VS Express 2005.
Both are WinXP systems with newest DirectX 9.0c available (including last patches).

The QuickStart_Windows.sln compiles fine. All references to XNA are found.

Now the strange thing.
Prototype.sln won't even compile because of 70+ errors occuring due to missing XNA references.
Each 'using' statement referencing Microsoft.Xna.* namespaces is underlined and declared as an error.
Hell, what's wrong?
This isn't the first time I have installed XNA alongside Express. I programmed a small game prototype half a year ago. Odd.

Edit:
Warning 80 Could not resolve this reference. Could not locate the assembly "Microsoft.Xna.Framework.Game, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6d5c3888ef60e27d, processorArchitecture=MSIL". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors. QuickStart.GUI

Does the bold part mean I need to install the old XNA framework 1.0 as well?
Dec 23, 2007 at 5:58 PM
I dont even know why the prototype is still in the source tree. You don't want to be using it. It was never converted to XNA 2.0, mainly because it was a demo that served its purpose and is no longer used.

You'll want to stick with the QuickStart_Windows/Xbox.sln solutions, and use the code in framework/code.


The black sky is fine, there is no skybox/plane rendering at the moment. Now the white terrain, that's a problem. If you could, go into Terrain.fx (in the QuickStartSampleGame project), and comment out the last line of the first pixel shader, replacing it with "return float4(1, 0, 0, 1);" If the terrain is now all red, it's a texture/shader problem and LordIkon will have to check it out. If it's still white, I'll have to do some digging around in the renderer.
Dec 23, 2007 at 6:29 PM
Changed the last line of the first ps method in .\framework\test\QuickStartSampleGame\Content\Effects\Terrain.fx according to your suggestion.
Terrain is now all red.

Concerning the prototype I take it that I can delete it from the folder I work with. Or I better just wait for an updated checkout version ... just to be on the safe side.
Dec 23, 2007 at 6:44 PM
Yeah, just ignore the prototype.

The terrain problem is interesting. I'm using an nVidia 8800 GTX on Vista 64 and do not experience that issue. In fact, when I have texture problems the whole terrain is black due to the way LordIkon's shaders work.

Are you using the latest change set? If you have the DirectX SDK installed, it may help to enable the DirectX Debug Runtime in the DirectX Control Panel and see if there are any error messages being emitted by the Debug Runtime.
Dec 23, 2007 at 6:52 PM
'cpc checkout QuickStartEngine' - used about an hour ago to get the latest changeset.
No DX SDK installed yet. Will do so later this evening.

Will try to report on this asap. But could be tomorrow before you hear of me again - family demands :)
Dec 23, 2007 at 8:38 PM
By the way, with the prototype, I had that problem. Its vs looking for the 1.0 dll's when you have replaced them with the 2.0 referances. Go into the referance folders, delete all xna links, and re-add them. That should solve it. Ive got nothing to add on the shader issues though :)
Dec 23, 2007 at 10:14 PM
Why are people still using the prototype? Is there a message on the main page that says "use the prototype for all new work" or something that I'm missing?
Dec 24, 2007 at 10:21 AM
Im not using the prototype, but i new of the bug from when i first joined. It was never fixed as we moved on to the current engine rebuild, so there was little point. I just thought id mention it as its useful to see what has changed over the course of the project.
Dec 24, 2007 at 3:09 PM
Edited Dec 24, 2007 at 3:13 PM
As I said here's a small status report on the DX SDK thing:

  • set the debug level within the control panel to maximum
  • activated the debug mode of DX (instead of retail)
  • activated most of the optional debug settings

To no avail - there were no errors DX was aware of.

So I tried PIX - as a first time user of this tool I had no real clue of what mindblowing things I was about to see.
Still got to get a hang on it... nearly 30 megabytes of log data - for one screenshot.
Anyway, I thought one of you might be able to parse through this pile of data and know what to make out of it.
So here they are. 2 files - one showing the data pushed through my graphics card with the pixel shader working.
The other showing the data when the ps is set to return 'red' for the terrain color.

http://rapidshare.com/files/78775532/BlackWhite.rar.html
( http://rapidshare.com/files/78775533/BlackRed.rar.html )

Edit:
And concerning the 'prototype' I might say: Out of curiosity.
The front page still says there's a prototype in development. And within the same text section the replacement of the old engine with a newer one is mentioned. So this might be the missleading trail so many tend to follow.
Dec 25, 2007 at 5:46 AM
Regarding the debug output, it seems like Visual Studio is unable to capture the DirectX debug output when using the managed debugger. Only the native debugger seems able to catch it. You will need to use an external kernel debugger like DebugView (on MSDN) to get the log output. Even with no errors/warnings, you'll get output from the DirectX runtime.

I'll take a look at those PIX dumps either Tuesday or Wednesday. I'm currently with family and have no access to any dev tools.
Dec 27, 2007 at 3:36 PM
Edited Dec 27, 2007 at 3:36 PM
Have you modified any of the source/shader files?

The problem is the final color exceeds white (1, 1, 1, 1) and is truncated to white. Looking at the hardware registers for in the incoming shader parameters, all of the parameters are seemingly ten times what they should be. i.e.,

DiffuseColor    (1, 1, 1, 1) -> (10, 10, 10, 10)
AmbientColor    (0.3, 0.3, 0.3, 1) -> (3, 3, 3, 10)
LightDirection  (0.5, -0.2, -0.5, 0.0) -> (5, -2, -5, 0)

The cause of this problem, I have no idea. Unless you modified the source during debugging for something. That's an odd one. :)

Dec 27, 2007 at 5:58 PM
Well, I didn't change anything - downloaded the source, hit debug and got the strange black&white problem.
Did you try downloading and debugging the current changeset yourself? Maybe something got messed up when it was uploaded to the server?
But in that case it wouldn't only be me seeing these strange results.
Dec 27, 2007 at 6:37 PM
I can verify that everything works as expected on my nVidia 8800 GTX on the following operating systems:

  • Windows Vista 64 with ForceWare 169.12
  • Windows XP Pro 32 with ForceWare 163.75

Let me update my XP drivers and try again.

What ForceWare drivers are being used when this problem occurs?
Dec 27, 2007 at 7:04 PM
ForceWare 169.21 on Windows XP Pro 32 also works.

Side Note: For some reason, the 169.21 installer on Windows XP royally screwed up something. After installing the drivers the first time, the desktop was locked at 640x480 in 4-bit color (yes, four-bit color). I had no idea such color depths even existed on graphics hardware after 1984. If you thought trying to use XP in an 8-bit palettized color mode was bad, try 4-bit! It looked worse than Windows 3.1!


Dec 27, 2007 at 7:19 PM
Driver version 6.14.11.6921 after installing latest forceware driver package on WinXP with SP2 and all available updates.
Doublechecked the graphics settings and made sure that each and every 3d application can freely take control over the graphics hardware. Which didn't help.
Maybe it is something with XNA 2.0?
Will try to install Refresh later this evening and see, if I can put the terrain shader to use in a 1.1 test app.
Coordinator
Dec 27, 2007 at 7:31 PM
If you can run version 0.182b from the template with no problems then it likely isn't XNA 2.0. In fact, the terrain in the new engine doesn't have anything specific to 2.0, at least not that I put in.
Dec 28, 2007 at 11:03 AM
Edited Dec 28, 2007 at 11:06 AM

LordIkon wrote:
If you can run version 0.182b from the template with no problems then it likely isn't XNA 2.0. In fact, the terrain in the new engine doesn't have anything specific to 2.0, at least not that I put in.


Where would I find this template if it was meant to be shipped with the changeset I downloaded?
The whole changeset folder is crowded with several versions of the engine - at least it looks like there are more than just the Quickstart_Windows/Xbox, Prototype and Template solutions in there... hidden in sub folders of the root folder.

In case you meant the zipped 0.182b version of the engine one can easily download from the main page of the project - verified, I can run that one. No terrain glitches, nice physics...
Screenshot: http://home.arcor.de/merc_artax/pictures/QSEngine_182b.JPG
Coordinator
Dec 28, 2007 at 4:20 PM
The template version in the versioned source download is old, I will eventually update that to 0.182b. But if you can run 0.182b fine then I don't believe XNA 2.0 specifically is your problem, as 0.182b uses XNA 2.0. It is likely something to do with the integration of the new graphics system and the new terrain system in the new engine.
Dec 28, 2007 at 4:47 PM
What's odd here is that all of the shader parameters are 10 times what they should be, according to the PIX dump. How that happens, I have no idea. If the shader was getting garbage data, or missing data, I could see it being a problem with assigning the values. But the values are consistently 10 times what they're supposed to be.

It doesn't seem to be a driver issue, as the code runs fine for me on Win XP with ForceWare 169.21 on an nVidia 8800 GTX.
Coordinator
Dec 28, 2007 at 5:20 PM
I should add that everything runs fine for me on Windows Vista Business and an nVidia 7900gs go. I am not at home right now so I'm not sure what ForceWare version I am using.
Dec 29, 2007 at 5:21 PM
Redownloaded the current changeset... just to be sure I didn't mess up anything. But the terrain shader still makes it all white.
When changing the shader to this piece of code (see below) I can see the multitextured terrain... without lighting, of course:

float4 MultiTexturedNormaledPS(VS_OUTPUT input) : COLOR0
{
float3 TerrainColorWeight = tex2D(TextureMapSampler, input.texCoordNoWrap);

input.Normal = normalize(input.Normal);

float3 normalFromMap = (2.0f * tex2D(SandNormalSampler, input.texCoord) - 1.0f) * TerrainColorWeight.r;
normalFromMap += (2.0f * tex2D(GrassNormalSampler, input.texCoord) - 1.0f) * TerrainColorWeight.g;
normalFromMap += (2.0f * tex2D(RockNormalSampler, input.texCoord) - 1.0f) * TerrainColorWeight.b;
normalFromMap = normalize(mul(normalFromMap, float3x3(input.Tangent, input.Binormal, input.Normal))) * 0.5f;

// Factor in normal mapping and terrain vertex normals as well in lighting of the pixel
float lightingFactor = saturate(dot(normalFromMap + input.Normal, -LightDirection));

float4 Color = tex2D(SandTextureSampler, input.texCoord) * TerrainColorWeight.r;
Color += tex2D(GrassTextureSampler, input.texCoord) * TerrainColorWeight.g;
Color += tex2D(RockTextureSampler, input.texCoord) * TerrainColorWeight.b;

float3 Reflect = (lightingFactor * input.Normal) + LightDirection;
float3 specular = pow(saturate(dot(Reflect, -CameraForward)), SpecularPower);

*//Color.rgb *= (AmbientColor + (DiffuseColor * lightingFactor) + (SpecularColor * specular * lightingFactor)) * AmbientPower;*
Color.a = 1.0f;

return Color;
//return float4(1, 0, 0, 1);
}

Screenshot: http://home.arcor.de/merc_artax/pictures/QSEngine_noLights.JPG

Doesn't look any different to me than the 0.182b code:

float4 MultiTexturedNormaledPS(VS_OUTPUT input) : COLOR0
{
float3 TerrainColorWeight = tex2D(TextureMapSampler, input.texCoordNoWrap);

input.Normal = normalize(input.Normal);

float3 normalFromMap = (2.0f * tex2D(SandNormalSampler, input.texCoord) - 1.0f) * TerrainColorWeight.r;
normalFromMap += (2.0f * tex2D(GrassNormalSampler, input.texCoord) - 1.0f) * TerrainColorWeight.g;
normalFromMap += (2.0f * tex2D(RockNormalSampler, input.texCoord) - 1.0f) * TerrainColorWeight.b;
normalFromMap = normalize(mul(normalFromMap, float3x3(input.Tangent, input.Binormal, input.Normal))) * 0.5f;

// Factor in normal mapping and terrain vertex normals as well in lighting of the pixel
float lightingFactor = saturate(dot(normalFromMap + input.Normal, -LightDirection));

float4 Color = tex2D(SandTextureSampler, input.texCoord) * TerrainColorWeight.r;
Color += tex2D(GrassTextureSampler, input.texCoord) * TerrainColorWeight.g;
Color += tex2D(RockTextureSampler, input.texCoord) * TerrainColorWeight.b;

float3 Reflect = (lightingFactor * input.Normal) + LightDirection;
float3 specular = pow(saturate(dot(Reflect, -CameraForward)), SpecularPower);

Color.rgb *= (AmbientColor + (DiffuseColor * lightingFactor) + (SpecularColor * specular * lightingFactor)) * AmbientPower;
Color.a = 1.0f;

return Color;
}
... very odd (how do I mark text sections as code?)...
Dec 29, 2007 at 6:06 PM
To create code sections, use double curly braces:

{ { code } }  Without the spaces between braces.


It's the AmbientColor, DiffuseColor, SpecularColor, and AmbientPower variables that are coming in 10 times what they should be.
Dec 29, 2007 at 10:16 PM
Edited Dec 29, 2007 at 11:06 PM
Mhm... got an idea... I am using a European (German) version of Windows XP.
We use commata to indicate floating point values.
Looking at the material file for the terrain I see you are using dots to indicate the beginning of the ... uhm, forgot the word, but you get the point.
Maybe you are using some .NET functions to parse that XML file which do have different behaviors/results depending on the culture of the operating system?
I'll try to verify this idea.

Edit:
I could narrow it down to the XML parser - when the material is passed as a binary file (xnb) to the reader the float values are already 10-times as big as in the XML file.
From what I saw I guess float.TryParse() in Material.cs might be the problem - it tries to interpret the text strings culture specifically.
Reading the documentation right now... perhaps this function can be told to always react like it was used on an English/American/International operating system.
Dec 29, 2007 at 11:18 PM
Interesting, I didn't even think about that. Good job finding the problem! I'm guessing it just ignores the periods and treats 1.0 as 10. I'll work up a quick patch and submit it as soon as I figure out how to force the locale. The affected files will be Material.cs as you mentioned, and also CompiledMaterialImporter.cs in the content pipeline assembly.
Dec 29, 2007 at 11:32 PM
Patch 635 has been uploaded. Try it out on your system and see if it fixes the problem.
Coordinator
Dec 29, 2007 at 11:33 PM
Wow, suzume, your screenshot just made me realize that the texture splatting is not matching up with the heightmap. It is reversed on either the X or Z axis. This is probably due to the coordinate system change. This might explain why the normal mapping is not working properly.
Dec 29, 2007 at 11:42 PM
Works like a charme, woohoo!
Screenshot: http://home.arcor.de/merc_artax/pictures/QSEngine_fixedFloats.JPG (give me a second or two to upload it)

@LordIkon: Didn't even notice it was not matching up... I'm just happy to see something on screen, at last! :D
Coordinator
Dec 29, 2007 at 11:48 PM
Lol, that is awesome, and guess what, the texture splatting is lining up again in your second screenshot, which means that you fixed two bugs related to using a language that uses commas in place of periods.

Notice where the sand texture was in your first screenshot versus the second? The sand was supposed to be at the bottom of the 'river'. River that would be there if we had water.
Dec 29, 2007 at 11:58 PM
Edited Dec 29, 2007 at 11:59 PM
Real glad I was of help, though shaw did the final trick... and I don't think the UV error has gone, yet. I took a shot from another angle and modified the picture using MS Paint (tm) to illustrate which direction might need to be flipped.
See: http://home.arcor.de/merc_artax/pictures/QSEngine_wrongUV.JPG

We don't want to overestimate commas, do we? ;D
Dec 30, 2007 at 12:02 AM
Edited Dec 30, 2007 at 12:04 AM
Glad that's been resolved. I'll submit it as a change set, unless anyone disagrees?

I can't comment on the texturing. My internet connection is messing up. My ISP is having issues with one of their connections to a top-tier router, so I have limited internet. Luckily, codeplex.com is accessible. home.arcor.de is not, unfortunately. It better be fixed soon!

I don't see anything in the XML material file that would reverse UVs if parsed incorrectly. Does the same happen on your machine, LordIkon, or is it just Suzume's screenshots?

I have a feeling Sturm isn't going to like the new float.TryParse blocks reaching a line length of almost 800 characters in Material.cs and CompiledMaterialImporter.cs ;)
Coordinator
Dec 30, 2007 at 2:26 AM
The UV flip is likely in terrain.fx shader, I think I flipped something wrong when I converted the coordinate systems. I'll get that fixed on Jan. 2nd :)
Dec 30, 2007 at 3:06 AM
Slacker! ;)

I didn't hear any objections (Sturm, you there?) so I committed the change set for the localization fix. I suppose I shouldn't make a habit of committing my own patches, but oh well...
Dec 31, 2007 at 10:43 PM
I'm not able to verify any new code for then next couple of days, so just feel free to submit. I'm going to do a review of the code once I get the change to do it, so expect a lot of code issues once I get to it. And if LordIkon updates the code guidelines you might not get as many :)
Coordinator
Jan 29, 2008 at 5:44 AM
When I run BE, which is the one from the newest source, I get an error:

\Utilities\Bin\x86 was unexpected at this time.

And then it brings me back to the command prompt.
Jan 29, 2008 at 6:22 AM
Can you reopen the issue and attach a screenshot of the prompt?

If you can it would be nice if you could go into \build\base and in the .cmd files change the first line @echo off to @echo on reopen the prompt and send me a screenshot of that.
Coordinator
Jan 29, 2008 at 6:42 AM
I emailed you the screenshot. I'll reopen the issue and attach it there as well.
Jan 29, 2008 at 7:45 AM
The reason for the failure is that you have () in your path settings, these do not go well on the command line, you need to add "" around the path setting for directx.
Feb 3, 2008 at 12:19 PM
Edited Feb 3, 2008 at 12:23 PM
Ive just done a fresh checkout, adn run setup.cmd. CLicked the shortcut and it then did the checks. Nunit was missing, along with physx, so i went to the relevant sites and downlaoded the physx driver and the .net 2.0 Nunit msi. It now reports physx as installed, but nuint still missing.

I alos get an error for utest, telling me to reinstall from the root with setup /snippets. This gives an error saying :

'cecho' is not recognized as an internal or external command... etc

ANy suggestions?
Feb 3, 2008 at 12:36 PM
Sounds strange do you get the other screen output in color? I'll have to look into it later as I'm not on my dev machine.

Regarding the Nunit error, did you dl the 2.4.3 version of the framework or a different one?
Feb 3, 2008 at 1:03 PM
Ah, I downloaded 2.4.6. Is that a problem?

The output was in colour, but the problem has fixed itself now. WHen I run BE, it all ok, save for nunit. Odd.
Feb 3, 2008 at 2:24 PM
The problem is that I check for the specific version, as there could be changes in the nunit model so I can't garentue that the unittests will run. So you need to dl the right version in order not to get the error. I will add the version number to the message, and I'll try to upgrade my nunit version and the script.