Twisted Metal Alliance: Temporary Forum

Aced14

Admin
Server Emulation

Offline

Posts: 134

Re: server emulation
Reply #15 - 01/05/14 at 17:06:55

DARKFORCE wrote on 01/05/14 at 16:28:16:

Awesome work. I had figured you would make quick progress. I'm really interested to see how you deal with the user log in issue (and account creation). If I'm not mistaken, I think the servers wrote information to your memory card every time you logged into it. Anyway, we're probably a while away from that.

The packet filtering thing wasn't my idea, *cough* Aced *cough*
There are Iris captures that *should* be 100% complete, but its logs are incompatible with all other packet capture/viewing applications.

Regarding the dual NTSC/PAL captures in a single file, it's probably from a single copy of the game. From my recollection, the cheat codes that allowed NTSC users to login to the PAL TMBO server (and vice versa) were capable of toggling the selected server via pressing select in the main menu.

Onto missing frames, when I used to trace in TMBO, the data I cared about was usually found within larger frames, so to more easily trace in real time I set filters for minimum frame sizes. DarkForce and Ven0m probably did the same as well, which would explain why some frames may be missing from the captures you've been looking at.

I also used to save my TMBO captures, but unfortunately the HDD they were stored on died at some point. Anyways, I was probably more agressive at setting frame filters than DarkForce.. I don't think I generally used very descriptive, action-based file names in mine...

Something else worth mentioning is that the NTSC version of TMBO doesn't use DNAS (at all), but the PAL version does... Not sure if (decomissioned?) Sony or game-specific servers are involved in authenticating DNAS, but if they are, that could pose additional challenges for the PAL version. All (or the vast majority) of DarkForce's saved captures are from the NTSC version.

Back to top

IP Logged

-SCoLD- Admin Offline Everybody dies, but not everybody lives. Posts: 1534	Re: server emulation Reply #16 - 01/06/14 at 13:16:06 Great news, No. 23. Thanks for sharing. When this becomes live again, I guess I'll be buying another PS3 with backwards compatibility. I would do it for Black: Online alone but the ability to play Outbreak again seals the deal for me. Good luck on your endeavor, looking forward to your first milestone!
Back to top	---===Reply of the Week===--- ----===The Tyrant of Twisted===--- ----==="The Link" Of The Week===--- IP Logged

No23

Superhero Accountant
Server Emulation

Offline

Posts: 52

Re: server emulation
Reply #17 - 01/06/14 at 14:32:37

Aced14 wrote on 01/05/14 at 17:06:55:

Regarding the dual NTSC/PAL captures in a single file, it's probably from a single copy of the game. From my recollection, the cheat codes that allowed NTSC users to login to the PAL TMBO server (and vice versa) were capable of toggling the selected server via pressing select in the main menu.

Ah, that's a possible explanantion.
The other packets seem to be pretty identical, so having NTSC and PAL users on one server is likely doable.

Back to top

---===TMBO Revival Expert===---

IP Logged

Aced14

Admin
Server Emulation

Offline

Posts: 134

Re: server emulation
Reply #18 - 01/06/14 at 23:42:06

No23 wrote on 01/05/14 at 13:20:18:

I had a quick look into some of the captures today. First of all I have to say that it's good you took a lot of similar captures because (at least those I opened) they are missing sometimes frames of data. Also there seems to be captures of a PAL game together with NTSC game in one file, this makes it a little bit difficult to distinguish between them.
We have the luck that the packets are not encrypted and the protocol pretty straight forward.

The first packet your server has to handle is a request for terms of usage. It's the same for the NTSC and the PAL version, just the text your server has to deliver is different.

Here's the generic packet structure:
2 bytes length of payload
2 bytes command
x bytes payload

Code:

first packet from PS2:
00000000  00 49 00 01 06 3f 01 00  2f 27 00 00 00 00 00 00 .I...?.. /'......
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 ........ ........
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 ........ ........
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 ........ ........
00000040  00 00 00 00 00 00 00 00  00 00 00 00             ........ ....

server's answer:
    00000000  0a 22 01 01 48 35 33 31  31 2e 31 00 00 00 00 00 ."..H531 1.1.....
    00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 ........ ........
    00000020  00 54 57 49 53 54 45 44  20 4d 45 54 41 4c 3a 20 .TWISTED  METAL:
    00000030  42 4c 41 43 4b 28 28 74  6d 29 29 20 4f 4e 4c 49 BLACK((t m)) ONLI
    00000040  4e 45 20 55 53 45 52 20  41 47 52 45 45 4d 45 4e NE USER  AGREEMEN
    00000050  54 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20 T               
    00000060  20 20 20 20 20 20 0d 0a  0d 0a 50 4c 45 41 53 45       .. ..PLEASE
    00000070  20 52 45 41 44 20 54 48  45 20 45 4e 54 49 52 45  READ TH E ENTIRE
    00000080  20 41 47 52 45 45 4d 45  4e 54 20 41 4e 44 20 49  AGREEME NT AND I
... and so on

I guess it's time to write a basic socket listener and deliver that packet Wink

Is it just me, or does the payload length appear to be off by a byte in the first packet the PS2 sends?

The entire packet seems to be 4C bytes long. If the payload length's 2 bytes were to be excluded, that'd make 4A bytes, whereas if both the length and command's combined 4 bytes were excluded, there'd be 48 (HEX) leftover bytes.

Could it be that the command is actually just 1 byte?

Back to top

IP Logged

GeneralChaz9 Spawn of Billy Mays Offline Posts: 200 Indiana, US Gender:	Re: server emulation Reply #19 - 01/07/14 at 15:23:46 You guys are awesome! I'll be following this closely, and good luck on getting it up.
Back to top	"They say that the mind bends and twists to deal with the horrors of life.... sometimes it bends so much it snaps in two." PSN: GeneralChaz9 Add me, especially if you want to play Twisted Metal! IP Logged

No23

Superhero Accountant
Server Emulation

Offline

Posts: 52

Re: server emulation
Reply #20 - 01/07/14 at 16:41:24

Aced14 wrote on 01/06/14 at 23:42:06:

The entire packet seems to be 4C bytes long. If the payload length's 2 bytes were to be excluded, that'd make 4A bytes, whereas if both the length and command's combined 4 bytes were excluded, there'd be 48 (HEX) leftover bytes.

Could it be that the command is actually just 1 byte?

I agree, guess I shot too fast Cheesy

Looking at some other packets it is more likely that the generic structure is
- 1 byte command
- 2 bytes payload length
- x bytes payload

Back to top

---===TMBO Revival Expert===---

IP Logged

Aced14

Admin
Server Emulation

Offline

Posts: 134

Re: server emulation
Reply #21 - 01/07/14 at 23:56:30

No23 wrote on 01/07/14 at 16:41:24:

I agree, guess I shot too fast Cheesy

Looking at some other packets it is more likely that the generic structure is
- 1 byte command
- 2 bytes payload length
- x bytes payload

I'm glad it wasn't just a matter of me going crazy while staring at that data, lol.

That revised structure makes sense to me. Checked some more login packets earlier and they seem to be in line with that. You probably already know this, but the 2 byte payload length is in Big Endian (right to left by order), which seems kind of strange IMO since the rest of the data (like the strings) are in Little Endian (left to right byte order).

Back to top

IP Logged

No23

Superhero Accountant
Server Emulation

Offline

Posts: 52

Re: server emulation
Reply #22 - 01/08/14 at 18:40:41

I wrote a 20 line testserver in python to see what happens if I replay the the "terms of use" packet. And it simply says "Connection to TMB:O server failed". After some thinking I finally realized what might be wrong. Aced14 wrote about setting the frame filter when he captured. This means that packets with small frames are missing (this also explains why half of the dns queries are not in the files). The consequence is that I replayed the wrong packet...

The problem is now that even the small packets are needed for a server recreation. I poked in several of the .ncf files without finding the first answer to command "00".

I hope that the IRIS captures are more detailed but Iris doesn't work with Windows7, so I have to work around that first Sad

Back to top

---===TMBO Revival Expert===---

IP Logged

DARKFORCE

Server Emulation
Offline

Posts: 264
Gender: male

Re: server emulation
Reply #23 - 01/08/14 at 19:49:12

No23 wrote on 01/08/14 at 18:40:41:

I hope that the IRIS captures are more detailed but Iris doesn't work with Windows7, so I have to work around that first Sad

I've started looking through all 600+ Iris capture logs. I'll let you know if I find one that contains the initial server connection.

Edit:
Found one

capture_Feb_02_2005_16_27_59_187.cap
@ 23:25:01:015

Hopefully it has what you need.

Back to top

« Last Edit: 01/08/14 at 21:06:40 by DARKFORCE »

TMBO Server Lobby Website

---===TMBO Video Historian===---

IP Logged

No23

Superhero Accountant
Server Emulation

Offline

Posts: 52

Re: server emulation
Reply #24 - 01/09/14 at 18:12:01

DARKFORCE wrote on 01/08/14 at 19:49:12:

I've started looking through all 600+ Iris capture logs. I'll let you know if I find one that contains the initial server connection.

Edit:
Found one

capture_Feb_02_2005_16_27_59_187.cap
@ 23:25:01:015

Hopefully it has what you need.

Looks promising.
I thought the first packet would request the terms of use, but it seems more like a session number is exchanged and then a "0b" packet requests the terms. I'll refine the little testserver and post the progress.
Because Iris didn't run with Win7 or Linux(Wine) I had to write a little tool to make the dumps readable in wireshark. It basically writes a hexdump textfile of the capture which in turn can be imported with wireshark. The only thing missing are the timestamps but those aren't that important.

Back to top

---===TMBO Revival Expert===---

IP Logged

RoaDiE Server Emulation Offline Cheesin' it up, OG style. Posts: 2592 Harper Land Gender:	Re: server emulation Reply #25 - 01/09/14 at 19:14:15 Lets say there are gaps in the packets... would it be hard to fill or just much much more time consuming?
Back to top	---===TM2PC Instructor King===--- IP Logged

DARKFORCE

Server Emulation
Offline

Posts: 264
Gender: male

Re: server emulation
Reply #26 - 01/09/14 at 19:18:39

No23 wrote on 01/09/14 at 18:12:01:

Looks promising.
I thought the first packet would request the terms of use, but it seems more like a session number is exchanged and then a "0b" packet requests the terms. I'll refine the little testserver and post the progress.

Awesome, glad it helped.

As for the session number... every player was assigned a session id when they logged into the server. The session id seemed to show up in chat packets and I believe it showed also up in the detailed view (when you pressed circle on a hosted lobby).

Back to top

TMBO Server Lobby Website

---===TMBO Video Historian===---

IP Logged

No23

Superhero Accountant
Server Emulation

Offline

Posts: 52

Re: server emulation
Reply #27 - 01/10/14 at 15:48:42

Heys guys,

took me some time to crawl through the captures and I am sad to say that most of them are useless Sad

Important packets are missing and it's a pain to put them together from several different files.

Anyway, I am optimistic that we can still do it. It's just not that easy as it looked in the beginning and I cannot promise that we won't hit a wall. Took me some days to find out how to handle the first few packets (my assumption about the command byte was wrong, the only thing that seems to be correct is the length of the payload, lol).

And this is the actual progress:

Yes, after several hours of digging I am confronted with Sony's terms of usage. We should change them.

Cool, nice virtual keyboard. I wonder what happens if I enter a fantasy account.

Of course the login fails ...

--- TO BE CONTINUED ---

Back to top

« Last Edit: 01/10/14 at 17:19:11 by No23 »

---===TMBO Revival Expert===---

IP Logged

DARKFORCE

Server Emulation
Offline

Posts: 264
Gender: male

Re: server emulation
Reply #28 - 01/10/14 at 16:30:12

That's horrible. I thought for sure the Iris packets would be complete. Aced strikes again! lol.

I believe that once we get past the login it should be pretty easy going from there.
I say that because the majority of the packet captures were of previewing and joining rooms. Hosted room packets should be there as well, though not as plentiful as the ones mentioned above.

Aced brought it to my attention a few days ago...

The live memory dumps I took of TMBO while the servers were running. Aced and I believe there are server status codes and other information that may be of use to you. Let me know if you'd like the file.

Back to top

TMBO Server Lobby Website

---===TMBO Video Historian===---

IP Logged

No23 Superhero Accountant Server Emulation Offline Posts: 52	Re: server emulation Reply #29 - 01/10/14 at 16:58:06 DARKFORCE wrote on 01/10/14 at 16:30:12: I believe that once we get past the login it should be pretty easy going from there. Let's cross fingers Btw., I wonder why the login password was transmitted as plain text without encryption
Back to top	---===TMBO Revival Expert===--- IP Logged

	Welcome, Guest. Please Login or Register