Skip to main content
Topic: [bug] makeobj r3354 - "random" crashes when merging (Read 5541 times) previous topic - next topic

[bug] makeobj r3354 - "random" crashes when merging

It seems there is some problem when merging files with wildcard, like this: makeobj merge target.pak *.pak The effects vary.

See first few posts for original description of problem:
http://forum.simutrans.com/index.php?topic=5142.0

After compiling makeobj and some experiments, it seems that there is some problem when merging:
(reply #22) http://forum.simutrans.com/index.php?topic=5142.msg50990#msg50990

I was able to get the Python caller to receive SIGSEGV, as it happens to Werner. Additionally, I had one case where makeobj started copying the file recursively to itself or something - the pak file grew at 100 MB/s.

So far I couldn't get anything to happen on Windows.

edit: This could have something to do with timing or environment, since I wasn't able to reproduce this outside the python script.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #1
Sorry, but did somebody work on this Problem??
I hope you understand my English

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #2
No / not yet.

If this problem happens only with specific paks, then please upload an archive of these paks somewhere.

Edit: small change in rev 3396 (use "delete []"). Maybe this lead to memory corruption?

If the problem is still there, please let me know. Extracting and merging the GUI.128.pak (from pak128 standard) worked fine for me (and valgrind, too).
Parsley, sage, rosemary, and maggikraut.

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #3
Werner: I would say it's a bit hard to debug a problem that is vaguely described and probably exists only outside debugger :P

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #4
But .. how we solf the problem?

At now I could use for pak128 an offizial makeobj, but there will be a new version and then .. whe have the problem :o(
I hope you understand my English

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #5
I looked through documentation for the python library and tried to find something to alter the execution... at least trying, you know ;)

After setting this on, I consistently get only the growing file, no signals.

If close_fds is true, all file descriptors except 0, 1 and 2 will be closed before the child process is executed. (Unix only). Or, on Windows, if close_fds is true then no handles will be inherited by the child process. Note that on Windows, you cannot set close_fds to true and also redirect the standard handles by setting stdin, stdout  or stderr.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #6
So .. there will be a new phyton script?

It would be on the svn too??
I hope you understand my English

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #7
It does not help... only the problem becomes consistent (for me, anyway).



edit: I can reliably reproduce the hang+infinite file this way:

1) pak128 r974
2) self-compiled makeobj r3397
3) #comment out all FOLDERS in top-level pakmak.tab except for "base" (no need to compile the whole set before)
4) when merging hangars, IT happens

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #8
sch ....

I test to use a spezial makeobj for PAK128...

At now I generate all Pnightly-PAKs with the same makeobj .. then PAK18 use an other makeobj than the other ....
I hope you understand my English

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #9
@VS: can you upload the offending paks that dont want to be merged?
Parsley, sage, rosemary, and maggikraut.


Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #11
Sounds to me rather that the created pak is found even while enumerating through all other paks. But then the enumeration of paks is done via the shell:

Code: [Select]
		if (!STRICMP(argv[0], "merge")) {
argv++, argc--;
try {
const char* dest = argv[0];
argv++, argc--;
root_writer_t::instance()->copy(dest, argc, argv);
}
...
return 0;

This means that there is already a pak with the tarbet name while adding with a wildcard?

(But a check for identical names would not be too hard to implement.)

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #12
Well... this is certainly one possibility. But on Windows it does not fail for some reason!



Here's the patch... it seems it fixes that.

Code: [Select]
Index: besch/writer/root_writer.cc
===================================================================
--- besch/writer/root_writer.cc (revision 3397)
+++ besch/writer/root_writer.cc (working copy)
@@ -301,7 +301,11 @@
  } else {
  find.search(argv[i], "pak");
  for (searchfolder_t::const_iterator i = find.begin(), end = find.end(); i != end; ++i) {
- any |= do_copy(outfp, root, *i);
+ if (strcmp(*i, name) != 0) {
+ any |= do_copy(outfp, root, *i);
+ } else {
+ printf("WARNING: skipping reading from output file\n");
+ }
  }
  }

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #13
Windows lock open file for writing. Maybe it just fails to be opened?

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #14
You're right. But that still leaves the question "how could this work before" ???

Werner: does the patch fix this problem for you?

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #15
I must test it tomorow ... at now I must sleep ;o)
I hope you understand my English

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #16
It work!

Now the Output is (at example):
Code: [Select]
Executing command: lift symbol.BigLogo.pak
Merging: makeobj QUIET merge GUI.128.pak *.pak
writing file GUI.128.pak
   copying file cursor.Mouse.pak
   copying file misc.Construction.pak
   copying file cursor.Marked.pak
   copying file cursor.GeneralTools.pak
   copying file symbol.Builder.pak
   copying file cursor.Builder.pak
WARNING: skipping reading from output file
My folders:  **none**
At this Part the makeobj without the Patch make an error ....
(But I don“t test the PAK, only the generation)

Edit:
When I count the quantity of the warnung I get 31, so it is Important for PAK128
Code: [Select]
simutrans@hedwig ~ $ grep "WARNING: skipping reading from output file" /home/simutrans/PAK/PAK128/info.txt | wc -l
31
I hope you understand my English

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #17
Dit it go into trunk??
I hope you understand my English

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #18
Warning will be generated for all folders since it's always merging against wildcard.

My projects... Tools for messing with Simutrans graphics. Graphic archive - templates and some other stuff for painters. Development logs for most recent information on what is going on. And of course pak128!

 

Re: [bug] makeobj r3354 - "random" crashes when merging

Reply #19
Testet and Solved in 3405

Thanks!!
I hope you understand my English