Couple things seem odd about the response.. ... SQL cluster because we don't own it
If you don't own the SQL servers then it doesn't sound like they have the RIGHT infrastructure. If a host can't provide you dedicated sql servers <plug for bitshop> then you should shop hosts </plug for bitshop>. .... Instance.... we won't be able to monitor it or cluster it for failure Again, instance or physical server - That shouldn't be an issue. Why can't you monitor it? We provide for some clients monitoring of sql, monitoring of locks (not deadlocks, normal locks, with variable time outs before notifications), monitoring that records are inserted at least every [x] minutes or notifications and more. There's plenty of hosts that know nothing more than how to run setup.exe, but I wouldn't think you are in that environment. If you are then you should use them for the infrastructure / hardware and have consultants / staff deal with the software on the boxes. I'm not a BizTalk expert, but it handles some things to do processing that may be appropriate. You may want to call your Microsoft rep and see if that meets this critical need. Steve Radich BitShop, Inc. - http://www.bitshop.com - IIS / SQL Hosting since 1996. -----Original Message----- From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of Falls, Travis D (HTSC, CASD) Sent: Friday, November 18, 2005 1:27 PM To: [email protected] Subject: RE: RE: [AspNetAnyQuestionIsOk] File still in use error Steve, I have no idea what you are talking about in the first half of your post all the way to the statement of "How off topic is this group getting if you actually read this?" As for the rest that was our/my point. I work in a place that provides the best technology, with survival critical real-time data processing applications and have the people and infrastructure to handle it. I happen to work for someone who does not agree and would rather serialize objects to a file stream the write to a SQL cluster because we don't own it. He also doesn't want me to write to our own instance on the server because we won't be able to monitor it or cluster it for failure. I found it on topic because I was looking for suggestions on how best to work around this architecturally... meaning MSMQ or simply program as Dave suggested with tons of catches with looking and waiting in side. Travis D. Falls | Consultant RAFT.Net IT | 860.547.4070 | [EMAIL PROTECTED] -----Original Message----- From: [email protected] [mailto:[EMAIL PROTECTED] Behalf Of Steve Radich Sent: Friday, November 18, 2005 1:01 PM To: [email protected] Subject: RE: RE: [AspNetAnyQuestionIsOk] File still in use error New hosting platform: ASP.Net 2.0 Visual Studio 2005 WAN conncetion provided by Schwinn Bicycles and Olympian wannabees. Just for those who want to write their SQL update statements as BizTalk file based processes to assure each update makes it to sql server. NOTE: Files sent by bicycle to Biztalk server. $10 from each account goes to support the Armstrong Foundation (HP? Laptops with Lance Armstrong exclusively used for hosting) How off topic is this group getting if you actually read this? There's a time to plan for outages, and a time to say that last 0.0___1% of uptime isn't cost effective to reach. The scale of cost effectiveness per second of uptime (measured on a yearly basis) starts sky-rocketing at some point. If you are working somewhere willing to pay for that last bit then maybe it makes sense to have a plan. I haven't read each message, but those I have read would make me think this isn't being pursued the best way. If they are worried about the sql going down then cluster it, or go with a reliable host that can run sql reliably. But when they reach that level plan on clustered servers, SANs, redundant web servers, redundant power supplies fed by 2 separate UPS systems, generators, redundant air conditioning, etc. If they have all those things then this is worth discussing. If they don't then those things then they aren't ready to worry about this kind of redundancy. For error logging: Run a sql server on the local box for logging errors and perhaps read only access to the db (in the event the main one is down). Isolate your database connections in code that if one fails logs it. (try/catch) In your failure write somewhere a log entry with the sql statement that failed, date/time, and error. Monitor this log. Even that's an overkill for most people, and most don't implement it. Steve Radich - http://www.ASPDeveloper.Net BitShop, Inc. - http://www.bitshop.com Managed Servers, Colocation, .Net Development (C#,VB,ASP) -----Original Message----- From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of Falls, Travis D (HTSC, CASD) Sent: Friday, November 18, 2005 12:46 PM To: [email protected] Subject: *AntiSpamTech.com* RE: [AspNetAnyQuestionIsOk] File still in use error ;-) that was pretty funny man; thanks I needed that... That was more help then any of the technical advice given the mood this has put me in. Travis D. Falls | Consultant RAFT.Net IT | 860.547.4070 | [EMAIL PROTECTED] -----Original Message----- From: [email protected] [mailto:[EMAIL PROTECTED] Behalf Of Dean Fiala Sent: Friday, November 18, 2005 11:40 AM To: [email protected] Subject: Re: [AspNetAnyQuestionIsOk] File still in use error This is piling on, but I can't resist. If the network is down, how can files be transferred? Do you have a fleet of bicycle couriers carrying floppy disks? :) On 11/18/05, Falls, Travis D (HTSC, CASD) <[EMAIL PROTECTED]> wrote: > He wants to be able to run if the database or network connection to the database goes down since the "business" of the application is file transfer... my take is if those things are down... they are your infrastructure... plumbing.... you shouldn't be expected to run without them. It doesn't help me become a better developer doing this stuff that is for sure. So Oracle isn't even an option... he would have the same network concerns... not for nothing but I work for a fortune 100 company... our network is fine. :-) > > Travis D. Falls | Consultant RAFT.Net IT | 860.547.4070 | [EMAIL PROTECTED] > > > -----Original Message----- > From: [email protected] > [mailto:[EMAIL PROTECTED] Behalf Of Dean Fiala > Sent: Friday, November 18, 2005 11:04 AM > To: [email protected] > Subject: Re: [AspNetAnyQuestionIsOk] File still in use error > > > Sigh. That has to be one of the top 10 instances of NIH (Not Invented > Here) syndrome I've encounterd. He'd rather you spend your time > solving a problem that a) doesn't need to be solved, or b) or doing so > t in a convoluted way instead of using an extremely mature and > reliable product. > > But sense no arguing with the insane. If he really mistrusts SQL > Server, have him pony up for Oracle or just be freakishly paranoid > with backing up the log files. Serializing an entire database because > you don't trust it is akin to buying a rickshaw to tow your Mercedes > because you are afraid the Mercedes might break down if you use it. > > This goes to #2, right after the company that built an entire > relational database engine in java, because they didn't think their > $100,000 Oracle installation was fast enough. > > Yes, they went out of business. > > On 11/18/05, Falls, Travis D (HTSC, CASD) <[EMAIL PROTECTED]> wrote: > > I am embarrassed to say this but my boss doesn't trust it. I have to read the entire database into an object structure and serialize it to the file system incase the database goes down. Most of my issues are due to him not trusting the infrastructure. He doesn't trust MSMQ as a state machine either. > > > > Travis D. Falls | Consultant RAFT.Net IT | 860.547.4070 | [EMAIL PROTECTED] > > > > > > -----Original Message----- > > From: [email protected] > > [mailto:[EMAIL PROTECTED] Behalf Of Dean Fiala > > Sent: Friday, November 18, 2005 9:29 AM > > To: [email protected] > > Subject: Re: [AspNetAnyQuestionIsOk] File still in use error > > > > > > Sounds very meaty indeed. One other thought comes to mind, why not > > use SQL Server to manage the state? It is very robust, supports > > transactions, and is perfect for keeping track of ticky-tacky details > > in an organized manner. > > > > On 11/18/05, Falls, Travis D (HTSC, CASD) <[EMAIL PROTECTED]> wrote: > > > I have been trying to research if it makes sense to change my state management from file system to MSMQ; I haven't seen much on state management out there though. > > > > > > Basically I have written a real time file transfer system using FileSystemWatcher that moves files from different nas locations to our local windows server (and vice-verca) and then executes a script in a custom procedural scripting language. > > > > > > For example, > > > if a file is found in location "X" meeting the requirement of say "*.pgp" it is copied to a specified destination(s) directory as "*.working" > > > I then write out the state file, > > > > > > I then rename it to "*.pgp" and write out the state file again, > > > I then delete the original and yes you guessed it... > > > write out a state file... > > > > > > the final step is kicking off our script that does PGP decrypt/encrypt, FTP, bla bla bla. > > > > > > It gets complicated if I have multiple files in a directory, and multiple destinations; say a destination and an archive.... I am just don't like my architecture for this and can't seem to think of a way to make it better... What I need to do is find another person that feels like doing a really harsh code review.... :-) Thanks for the help Dean... as always you are full of great suggestions. The system works great the issue is recoverability if the system goes down or there is an issue. This is my first real-time multithreaded application; before this I lived in the response-request paradigm of the web. > > > > > > Travis D. Falls | Consultant RAFT.Net IT | 860.547.4070 | [EMAIL PROTECTED] > > > > > > > > > -----Original Message----- > > > From: [email protected] > > > [mailto:[EMAIL PROTECTED] Behalf Of Dean Fiala > > > Sent: Friday, November 18, 2005 8:44 AM > > > To: [email protected] > > > Subject: Re: [AspNetAnyQuestionIsOk] File still in use error > > > > > > > > > Hmm, you have other service or process paying particular attention to > > > the file system. A virus scanner perhaps? > > > > > > You could also try sleeping the threads briefly, though that might > > > defeat the purpose. Are you sure you only have one instance of the > > > app running? I've scrod myself in similar fashion before. > > > > > > Another option, is as you suggested to use another queuing mechanism. > > > > > > Or, though this is a bit messy, you could version your files, and > > > clean up at your leisure. > > > > > > First time you save TDF02 in C:\RAFT.NET\Service\State\IDs\TDF02_1.state > > > then TDF02_2.state, etc. > > > > > > Your retrieval piece would just have to be smart enough to grab the > > > latest version of each object. > > > > > > > > > > > > On 11/18/05, Falls, Travis D (HTSC, CASD) <[EMAIL PROTECTED]> wrote: > > > > No it is off; thanks for the thoughts Dean. I am starting to think that the OS simply isn't releasing the files quick enough... > > > > > > > > Travis D. Falls | Consultant RAFT.Net IT | 860.547.4070 | [EMAIL PROTECTED] > > > > > > > > > > > > -----Original Message----- > > > > From: [email protected] > > > > [mailto:[EMAIL PROTECTED] Behalf Of Dean Fiala > > > > Sent: Friday, November 18, 2005 8:12 AM > > > > To: [email protected] > > > > Subject: Re: [AspNetAnyQuestionIsOk] File still in use error > > > > > > > > > > > > Go to the Services console. You can find it the Administrative Tools > > > > program group. > > > > > > > > There is a service called Indexing Service. Stop it and set its > > > > startup type = manual. > > > > > > > > On 11/18/05, Falls, Travis D (HTSC, CASD) <[EMAIL PROTECTED]> wrote: > > > > > Dean, > > > > > Thanks for the response... I have been struggling with this for weeks. I am currently in development stages and I am running this console application on my laptop. I don't know if indexing is turned on... how would I check? > > > > > > > > > > Travis D. Falls | Consultant RAFT.Net IT | 860.547.4070 | [EMAIL PROTECTED] > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: [email protected] > > > > > [mailto:[EMAIL PROTECTED] Behalf Of Dean Fiala > > > > > Sent: Thursday, November 17, 2005 10:29 PM > > > > > To: [email protected] > > > > > Subject: Re: [AspNetAnyQuestionIsOk] File still in use error > > > > > > > > > > > > > > > Travis, > > > > > Do you have the Indexing Service on, on this server? If so, turning > > > > > it offf, might solve the problem. > > > > > > > > > > On 11/17/05, Falls, Travis D (HTSC, CASD) <[EMAIL PROTECTED]> wrote: > > > > > > I have the following method in a utility class I created that is instantiated via a singleton. I think begin my file work load that can be 100 files... I start 10 threads that do a copy, rename, delete, script work load. During these threads I create files based on the work item's ID for example S053R.state with the method below. Every step of the way I call the below method to serialize an object to maintain state incase the system goes down or the transfer fails. > > > > > > > > > > > > My questions: > > > > > > > > > > > > 1. Why do I keep getting the: > > > > > > 2005-11-17 14:22:40,305 [2096] ERROR com.TheHartford.HTSC.RAFT.FileWatcher.Utilities [(null)] -EXCEPTION: Access to the path "C:\RAFT.NET\Service\State\IDs\TDF02.state" is denied. System.UnauthorizedAccessException: Access to the path "C:\RAFT.NET\Service\State\IDs\TDF02.state" is denied. > > > > > > at System.IO.__Error.WinIOError(Int32 errorCode, String str) > > > > > > at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, > > > > > > FileShare share, Int32 bufferSize, Boolean useAsync, String msgPath, Boolean bFromProxy) > > > > > > at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize) > > > > > > at System.IO.FileInfo.Create() > > > > > > at com.TheHartford.HTSC.RAFT.FileWatcher.Utilities.DoSerialize(String Path, String Name, Object o) in c:\raft\filewatcher\code\utilities.cs:line 438 > > > > > > > > > > > > 2. would it be better to create a state engine using MSMQ instead? > > > > > > > > > > > > Any help would be greatly appreciated; I have been struggling with this one for a while. I feel like I am closing everything and I know this is the only method and process that creates and deletes these files. > > > > > > > > > > > > > > > > > > public void DoSerialize(String Path, String Name, object o) { > > > > > > lock (syncRoot2){ > > > > > > try{ > > > > > > object j = new object(); > > > > > > j = o; > > > > > > DirectoryInfo di = new DirectoryInfo(Path); > > > > > > FileInfo[] finfos = di.GetFiles(Name); > > > > > > for(int i = 0; i < finfos.Length; i++){ > > > > > > FileInfo file = finfos[i]; > > > > > > try{ > > > > > > file.Delete(); > > > > > > } > > > > > > catch (FileNotFoundException fnfe){ > > > > > > log.Error("FILE NOT FOUND: " + fnfe.ToString() + " " + fnfe.Message); > > > > > > } > > > > > > catch (IOException fnfe){ > > > > > > log.Error("IO EXCEPTION: " + fnfe.ToString() + " " + fnfe.Message); > > > > > > } > > > > > > } > > > > > > FileInfo fi = new FileInfo(Path + Name); > > > > > > using (Stream myFileStream = fi.Create()){ > > > > > > SoapFormatter serializer = new SoapFormatter(); > > > > > > serializer.Serialize(myFileStream, j); > > > > > > DoGarbageCollection(); > > > > > > } > > > > > > } > > > > > > catch (Exception ex){ > > > > > > DoMessaging("FWE00017", ex.Message + " " + ex.ToString(), GetDateNowDateTime()); > > > > > > log.Error("EXCEPTION: " + ex.Message + " " + ex.ToString()); > > > > > > } > > > > > > } > > > > > > > > > > > > > > > > > > ************************************************************************ * > > > > > > This communication, including attachments, is > > > > > > for the exclusive use of addressee and may contain proprietary, > > > > > > confidential and/or privileged information. If you are not the intended > > > > > > recipient, any use, copying, disclosure, dissemination or distribution is > > > > > > strictly prohibited. If you are not the intended recipient, please notify > > > > > > the sender immediately by return e-mail, delete this communication and > > > > > > destroy all copies. > > > > > > ************************************************************************ * > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Dean Fiala > > > > > Very Practical Software, Inc > > > > > http://www.vpsw.com > > > > > > > > > > > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Dean Fiala > > > > Very Practical Software, Inc > > > > http://www.vpsw.com > > > > > > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Dean Fiala > > > Very Practical Software, Inc > > > http://www.vpsw.com > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Dean Fiala > > Very Practical Software, Inc > > http://www.vpsw.com > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > -- > Dean Fiala > Very Practical Software, Inc > http://www.vpsw.com > > > > > Yahoo! Groups Links > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > -- Dean Fiala Very Practical Software, Inc http://www.vpsw.com Yahoo! Groups Links Yahoo! Groups Links Yahoo! Groups Links Yahoo! Groups Links ------------------------ Yahoo! Groups Sponsor --------------------~--> AIDS in India: A "lurking bomb." Click and help stop AIDS now. http://us.click.yahoo.com/VpTY2A/lzNLAA/yQLSAA/saFolB/TM --------------------------------------------------------------------~-> Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/AspNetAnyQuestionIsOk/ <*> To unsubscribe from this group, send an email to: [EMAIL PROTECTED] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
