WriterAdvanced: writer not available?? dl0haus 10/28/2008 1:49:01 PM I have asked several questions regarding this however I thought I would change the angle of the question a little bit. I have an archival application that downloads live streams. We do a lot on weekends especially so its multithreaded. This application works well 99.9% of the time, however, sometimes it crashes. I have noticed sometimes that the writer object is NULL for reasons I cannot explain. When I am stress-testing the system I can download 140-160 simultaneously. On a busy night, we might have 90 going at once. If I do the math, assuming that each thread has its own context for disk access, etc, when we've got a lot running we are doing 2000-3000 seeks and writes PER SECOND to the hard drive. I am wondering if it is possible for the WMFSDK to be in a condition where it requests a write to the disk and Windows just doesn't have any at that moment and I miss the write in that case? Is there a condition or test available that maybe allows for this condition... to ask on that particular frame or set of frames you want to write that a writer simply wasn't available at the time, to bypass some code and wait for the next round of data? I would rather lose a few frames than crash or end up with some "bad state". Thanks, D

Re: WriterAdvanced: writer not available?? "Alessandro Angeli" <nobody[ at ]nowhere.in.the.net> 10/28/2008 4:45:32 PM From: "dl0haus" [Quoted Text] > I have an archival application that downloads live > streams. We do a lot on weekends especially so its > multithreaded. This application works well 99.9% of the > time, however, sometimes it crashes. I have noticed > sometimes that the writer object is NULL for reasons I > cannot explain. If by "writer object" you mean your reference to IWMWriter, than the only explanation is that your own code sets it to NULL, because that memory location is only known to your stack or your heap. > I am wondering if it is possible for the WMFSDK to be in > a condition where it requests a write to the disk and > Windows just doesn't have any at that moment and I miss > the write in that case? Windows doesn't have such a huge I/O bug that I'm aware of. Whether the WMWriter ignores some error condition returned by the OS and misses a write is a different story, but I've not read any other report of this happening. On the other hand, if you have a bug that sets the wrong reference to a writer to NULL, maybe the same bug calls EndWriting() on the wrong writer, too. You should also consider that the WMF runtime is probably not designed for this kind of scenario and writing your own MMS client that downloads the stream directly instead of remuxing it would be far more efficient. -- // Alessandro Angeli // MVP :: DirectShow / MediaFoundation // mvpnews at riseoftheants dot com // http://www.riseoftheants.com/mmx/faq.htm

Re: WriterAdvanced: writer not available?? dl0haus 10/28/2008 5:13:02 PM I figured that I would be coming up against some walls here with the functionality but I found that my own MMS, while it worked, needed some more work relative to disk access. The WMFSDK seems to be highly optimized in that you can write lots of different stuff and it isn't "linear" meaning it is tapped into the caching and disk management that Windows uses at a lower level. I can download 200 streams at once with this code. The only thing that I'm missing is there are these "glitches" that occur from time to time. I believe I have the "crash" I was experiencing solved... I force EOF when I see this. At this point I think all I need to do is find the various error conditions that can occur (whether they should or not) and force EOF. That is better than a crash and that allows our operations team to just restart the download and continue where it left off. I am not doing anything weird with the Writer and it goes null for reasons I cannot see. I do no buffer allocation or manipulation, I simply use the facilities provided in the SDK to do this stuff. I am not overwriting memory. I have CMutex sections around the reads and writes, however, I have converted the class to use all instance variables. I still want to protect that code. I have thread locking where its obvious. I don't believe that Windows has a huge I/O bug like that either. I'm thinking I'm pushing the limits of the SDK but it does a good job for what it does and it saves me from having to work with my own MMS implementation which would require a bit more work to get the buffering and writing as such to download 120 streams a night on a busy night. If you think about it I have managed to write a DVR (Tivo) that can do 120 stations at the same time without breaking a sweat... if I could only find these few glitches and trap all the error conditions to force EOF... Thanks, D "Alessandro Angeli" wrote: [Quoted Text] > From: "dl0haus" > > > I have an archival application that downloads live > > streams. We do a lot on weekends especially so its > > multithreaded. This application works well 99.9% of the > > time, however, sometimes it crashes. I have noticed > > sometimes that the writer object is NULL for reasons I > > cannot explain. > > If by "writer object" you mean your reference to IWMWriter, > than the only explanation is that your own code sets it to > NULL, because that memory location is only known to your > stack or your heap. > > > I am wondering if it is possible for the WMFSDK to be in > > a condition where it requests a write to the disk and > > Windows just doesn't have any at that moment and I miss > > the write in that case? > > Windows doesn't have such a huge I/O bug that I'm aware of. > Whether the WMWriter ignores some error condition returned > by the OS and misses a write is a different story, but I've > not read any other report of this happening. On the other > hand, if you have a bug that sets the wrong reference to a > writer to NULL, maybe the same bug calls EndWriting() on the > wrong writer, too. > > You should also consider that the WMF runtime is probably > not designed for this kind of scenario and writing your own > MMS client that downloads the stream directly instead of > remuxing it would be far more efficient. > > > -- > // Alessandro Angeli > // MVP :: DirectShow / MediaFoundation > // mvpnews at riseoftheants dot com > // http://www.riseoftheants.com/mmx/faq.htm > > >

Re: WriterAdvanced: writer not available?? "Alessandro Angeli" <nobody[ at ]nowhere.in.the.net> 10/28/2008 6:35:26 PM From: "dl0haus" [Quoted Text] > access. The WMFSDK seems to be highly optimized in that > you can write lots of different stuff and it isn't > "linear" meaning it is tapped into the caching and disk > management that Windows uses at a lower level. As fas as I can tell, the WMWriterFileSink used by the WWriter uses plain user-mode files, like any other application. > breaking a sweat... if I could only find these few > glitches and trap all the error conditions to force > EOF... Unfortunately you are the only one who can try to solve this, since it requires studying your own code. -- // Alessandro Angeli // MVP :: DirectShow / MediaFoundation // mvpnews at riseoftheants dot com // http://www.riseoftheants.com/mmx/faq.htm

Re: WriterAdvanced: writer not available?? dl0haus 10/28/2008 8:37:02 PM I think I will solve it I was just asking a general question about when the writer goes to disk if there are a pool of writers or something where its possible to be in a condition where you might need to have the current thread wait until one becomes available. I have no other explanation. I did find something interesting in the C# calling convention to the DLL that seems to have made the application more stable. Who knows, maybe that was the reason I was getting NULL writers occasionally. D "Alessandro Angeli" wrote: [Quoted Text] > From: "dl0haus" > > > access. The WMFSDK seems to be highly optimized in that > > you can write lots of different stuff and it isn't > > "linear" meaning it is tapped into the caching and disk > > management that Windows uses at a lower level. > > As fas as I can tell, the WMWriterFileSink used by the > WWriter uses plain user-mode files, like any other > application. > > > breaking a sweat... if I could only find these few > > glitches and trap all the error conditions to force > > EOF... > > Unfortunately you are the only one who can try to solve > this, since it requires studying your own code. > > -- > // Alessandro Angeli > // MVP :: DirectShow / MediaFoundation > // mvpnews at riseoftheants dot com > // http://www.riseoftheants.com/mmx/faq.htm > > >