Hard Drive Lag!

Someone must have an answer to my problem. Maybe this has been addressed before, and believe me I have searched and searched to no avail.

2 PCs, 2 different locations, one home, one at work. Not the biggest and best but from what I have read, adequate.

Drive induced lag:

Observed - In world, working in my workshop. Running between 17 -23 FPS, no packet loss, ping around 39 ms, Bandwidth running a steady 400-500. Everything is fine.

All of a sudden (maybe 45 minutes into being inworld) my hard drive starts caching, or crunching, or crashing or something, and it is over. Can't move, can't select, just have to wait for the drive to stop doing whatever it is doing. Can sometimes last for 3-5 minutes.

First of all, I can be in a Class 3 Sim or a Class 5 Sim, and experience the exact problem on either machine. Sometimes I can go days on end and never have this problem manifest.

Trust me, I have played with every preference setting again and again. Think it is resolved but it returns. I have tweaked and un-tweaked. I have experienced horrible packet loss at times but not have the problem described. I know it is not network related, because I have monitored everything again and again.

I can be standing in the same spot for 30 minutes, just working on a script, not moving then it starts. No textures loading, no updating of any sort and the drive goes berserk.

Now this happens at either location, different systems, different connection speeds, and such. I know it is not the PCs because they both can render a very satisfactory performance 60% of the time. It seems things get a lot worse as the Client is upgraded. 1.13 never had this problem. Won't even go into 1.14 (heh!).

What is being written to the drives and why? Shutting down? The drive will run and run and run.......5 minutes on logoff.

As far as connection speeds, the home is 630 Kb/s and work is 1604 Kb/s.

Is this a Virtual memory thing, cache problem or what? I have read many posts that seem to describe the same problem, Pulsing Lag etc. but excessive drive activity never seems to be mentioned (or possibly not noticed). Oh BTW, my hard drives are CLEAN and not fragmented, a ritual I follow continually. Both PCs have 768meg Radeon 9250 and Nivida GeForce 2 MX400, 39ms and 28ms ping time.

Any ideas?

Most likely, your computer is running out of memory, and so the OS is starting to swap the application to disk. That can get very bad, very quickly. The memory footprint of the SL client has increased over time, with a big jump when the First Look rendering code was introduced in 1.14, so a system that had just enough for 1.13 might not have enough for the current client. The client also consumes more memory as you remain in-world longer, because it continues to save textures and other data. (There are also reasons to suspect that the client has a memory leak; in other words, it consumes memory that it's not even putting to good use.)

The best solution is to buy more RAM; upgrade to 2 gigabytes if your system and budget will accept it. Stopgaps are shutting down all other programs while using SL, and getting rid of as many things as possible from the System Tray (Windows, but you can do the same sort of things on Linux and Mac). If you have a system that you use ONLY for SL (not web surfing, reading email, or anything else that goes online), you could consider removing antivirus software; that typically gets you back 50MB of RAM. A further trick for advanced users; disable any Windows services that you don't need. Doing all of those things might get back over 100MB of memory space, which will make a difference.

1.14 also introduced a new system for storing the disk cache. Generally performance of the new cache scheme seems to be better, but its brains get scrambled once in a while. The symptom is that your client will COMPLETELY freeze for seconds at a time. The only fix I have found is to clear the cache. I've had to do that about once every two weeks since I started using the 1.14-based First Look clients, and 1.15 seems to still have the problem.

There is a bug report dealing with this issue, do a search on Jira for memory leak. Seems that most having this problem (at least those who are reporting it there) have Nvidia graphic cards.

hi =)
this seems to be a memory leak. it´s allready reported a few times but hard to fix i think. only workaround is in my opinion to get at least 1.5 gig ram (it takes long before sl uses the 1.5 gig, maybe you need more under vista, i use xp)

at the moment i am trying to figure out if it´s possible to reduce this by playing with different grafic settings. i have 2 computers. one with an old grapics card, and there i don´t have the problem... maybe because i disabled the vertex shaders there or other options. i will post here when i find something

edit: i have this problem with my ati card, my other nvidia does not have the problem, so i think its not about nvidia or ati

BTW, try defragmenting your hard drive. In itself this won't cure the problem (maxing memory is the best answer, most likely), but it might reduce the severity.

And make sure you have plenty of free space on your hard drive.

TYSM for all of your quick and precise explaination of my problem. Yes, I know my system is a little on the skimpy side, but when it works, it works quite well.

Will LL be sending out additional RAM to us with future client upgrades? (hehe)
I can't help wonder how much RAM we will need next month.

At times I not only have SL running, but Gimp, Blender 3d, Firefox (4 or 5 tabs opened), email client and not experience any problems. That is quite a load for my system and it runs SL flawlessly. Other times, only SL is running and it turns to crap.

To be sure, this is a memory leak problem more than anything else, discussed here:

https://jira.secondlife.com/browse/VWR-364

It is obvious to me that this is affecting more people than we know about. My home system has a Raedon 9250 256meg and work is Nvidia GeForce2 MX400 and both experience the same problem with different graphic cards.

The jira thread link is very interesting to say the least, even if I don't understand what on earth is being said!

I will continue to monitor the jira thread and in the meantime, take some of your suggestions and cut down on the load on my system. I will pass on the 4Gig memory upgrade in the jira thread tho, or at least until the next client upgrade! heh!

Thanks!

Just to report:

I have 2 gigabytes of RAM and I experience the same swapping out issue with my PC. (But not as severe as the original poster of this thread.) I generally don't run any other applications while I'm running SL except the antivirus program, and some very minimal background applications.

If more RAM were the answer, I would not be able to load my system down as much as I do at times and be able to run the client efficiently.

How much RAM are we actually using and how much is being wasted?

From what I have read, and actually understood, this is not something new but goes back a long way. The client changes made recently are just making things worse and causing it to show its' ugly head faster.

Clear your cache and defrag your HD. That will reduce the disk thrashing. However it won't solve the memory leak :\

Its been said loads of times on these forums.. defrag your hard disc every day you play SL, run scandisk once a week and check for bad sectors and to refresh the disc filing system.. run a registery cleaner (your hard disc maybe searching for lost or broken links), turn off disc indexing (unless you do a lot of searches).. 2gb ram, higher grade GC nVidia 7 series or better with minimum 256mb ram (not shared).

I am seeing this problem too, i have 2 Gig of ram however I have an ATI card. So much for the Nvidia theory...

This definately sounds like it has to do with swapping (or as it's frequently called, thrashing - because it renders your comp useless for a while). When a certain amount of memory is used, and more is needed, your system will start putting parts of this memory on disk, and because disk is so much slower, this lowers performance when that particular part on disk is needed again, because it has to be fetched of the slower disk into faster ram again... It should normally do this first for processes who are not actively running though (eg when you have sl and a browser running, and not actively using the browser, it will swap out the browser first). When also your active process (here sl) get's swapped out (because otherwise you shouldn't see this performance penalty in the sl application, only when switching to eg the browser again) this means something is using up almost all of your memory, and asking for more fast.

More memory somewhat helps, because it delays the moment when the memory is almost all used up... but even better is to find the cause of this memory-usage and try to fix it... In the windows taskmanager you should be able to see which process is using up all your memory, by going to the processes tab, and sorting by memory usage. The process that is eating up all the memory will usually be the one with the highest memory usage there... In case this process is secondlife, then I suppose there's nothing much to do, but wait and hope for a better working client version... (or buying tons of ram )
defragging etc will help a bit, because it makes your harddisk work more optimal, but will not help to cure the root of the problem...and will probably not make enough of a difference to keep sl playable once the swapping starts!

Something you can check is whether your windows is optimized for desktop processes (good) or background processes (in this case - bad, only good for servers etc). This setting is hidden somewhere in control panel-->system (don't remembver exactly where, and can't check now)

hmm, long post, I hope it's not too confusing

And as is said almost not at all; disable the Indexer on your boot drive and any other drives with SL data on them, flush out any Java releases present except the very latest one, and consider buying a second hard drive to hang off your CD-ROM cable, to split the data up into parallel paths.

There does seem to be a problem with the top end Nvidia cards at present; I was astonished by a middle-aged Dell 1.8 P4 with an MX400 Nvidia, it ran way better than my 7300 Nvidia equipped 3.0 dual Xeon!

To swap between Desktop and Server styles for the OS in Windows PC, control panel - system - advanced tab - Performance button - visual effects. Change to 'Adjust for best performance. Advanced Tab - change settings Processor and Memory Usage to Performance.

I always change the 'look' of windows to Classic Style, removing smooth fonts, Smooth scrolling menus etc.

Certain virus checking programmes 'Norton' being one, continously checks your HD for virus's as you use the PC, they cause huge overheads with the SL Client, remove all folders in C:/Documents and Settings/<your logon name>/Application Data/Secondlife from the scan. But, make sure you scan those folders using a custom scan, regularily.

The reason I mentioned GC's, was simply the GC you currently use, although complies with the 'current' minimum specs for running SL, with the increases of the rendering pipelines in the new client and subsequent clients, you will see a degrading of the visual experience in SL. The minimum specs for running SL haven't changed over the years, but the client core software has in the past 3-4 months has. LL MUST update those specifications and stop deluding people what it takes to run SL these days, otherwise people will be misinformed.
I run a nVidia 7950 GT and that often gives me 35-100+ fps.

EDIT[ I also run 3 SATA II HD's and 1 IDE HD, giving me over 1.2tb space, I have placed the SL cache and the OS cache on seperate HD's, this also improves performance.

I have been monitoring Page Files and Available Free memory with Task Manager.
The system has been running very stable with approximately 238MB of available Free RAM.

At the very present time, SL is running perfectly while I am also running Firefox AND Gimp.

BUT: last evening, while everything was working like clockwork, only SL running, with Task Manager open, I watched the Available Free RAM suddenly drop like a rock from 206MB to 16MB this was within 5 seconds. I was standing in my workshop, had been inworld for about 4 hours, doing the exact same thing. Something grabbed nearly all of the available memory for about 1 minute then started releasing it slowly. I watched it build itself back up to around 200MB over a period of 3 minutes.

There was no disk thrashing but performance started degrading.

Another instance as the memory started getting eaten up, I TP'd to a Class 5 Sim, and the available Free Memory jumped to 325MB instantly. I then TP'd back to my Class 3 Sim, and the memory stayed at around 300MB and never had another problem.

Something is grabbing the memory and at times not releasing it or releasing it over a period of time. Currently Second Life is using 78MB of RAM and at times it will climb to 468MB for absolutely NO reason. I am in the same room, doing the exact same thing, barely even moving.

Memory leak!

There seems to be some connection as to what I am doing at the time, once when I selected Inventory, it started going nuts, but stabilized. I could not repeat the incident.

I have optimized Windows for performance as AWM Mars suggested, have run without explorer and minimum services. There is no appreciable difference noticed in Task Manager. If you examine the jira thread, you will see that Torley Linden is even experiencing the exact same problems. RAM is not the answer, tweaking this and tweaking that, has little effect.

There are serious problems in the SL client that has nothing to do with texture caching, drive fragmentation, RAM size, etc.

We are only putting a larger bucket under the leak.

Right now SL is FLYING for me. I haven't changed a thing.

Take a close look at this:

http://i170.photobucket.com/albums/u266/yrrek_gran/screenshot.jpg

There is a lot of stuff running and SL client is stable and running like a champ!
Plenty of Free RAM

Looks good..... but only one client ??? I ran 4 on one PC

hehe!

One client working correctly is good enough for me.

Just wanted everyone to see that there is plenty of RAM to successfully run the client and still have plenty of memory to multi-tasking with only 768MB of RAM.

When the SL client is working properly, you don't need terabytes of RAM, just stable software.

I will continue to monitor this and maybe I will get lucky and be able to pinpoint an action that might cause this.

In the meantime, those of you that are experiencing this same problem, I suggest that you keep the Task Manager open and monitor the Physical Available Memory. If it starts dropping rapidly, make some notes of what you were just doing and maybe we can figure out what is causing this.

You are right... The crash logger doesn't seem to work for everyone. but if it did, rather than see it as an inconvience, people actually used it, then it would be easier for the coders to see what parts of the programme are effecting which particular combinations of setups. The more information the better quality of bug fixing.

Perhaps the first 'bug' that needs fixing IS the crash logger? Maybe even make it transparent.

Yrrek, the memory used by your client also depends on stuff you don't see, happening nearby. How "nearby" is depends on your prefereces graphics distance setting.

I don't dispute the claim that there may be a memory leak. It's just not as easy to demonstrate it as your evidence.

Note that "Available" memory isn't the best value to watch, but "Mem Usage" in the Processes tab. To see that, use "View -> Select Columns". I also like to watch "Peak Mem Usage" and "VM Size" when tracking these kinds of issues.

The reason you don't want to watch Available memory (or rather, Performance -> Physical Memory -> Available" is that it really doesn't show the amount of free memory. You see, a program can "free" memory but that memory often isn't actually put back into the available pool by the system until some other event happens forcing it. Trust me, this indicator can be very misleading. MS could make the indicator better, but it would probably reduce performance. Virtual memory (disk cacheing) complicates things further, because your computer is often using more memory than it has. (At least, the processor thinks it is, and the VM paging system fools it into working.)

Generally, the causes for thrashing (which has the symptoms described by the OP, although other things can do that too) are these:

1) simply trying to do way more than your poor computer can handle (using far more memory than it has). This is usually not the case because the default swap file size is only twice the physical memory.

2) poor programming by the application programmer, causing too much memory to be accessed in rapid succession, so that the system has to keep paging memory in and out of the disk. I doubt this is the case here, but it could be on a system with very limited memory.

3) running too many applications that all access lots of memory constantly.

4) operating system bugs

It also matters whether your video card uses system RAM rather than its own RAM; I'm a bit out of date but I assume all the cards we talk about here have dedicated video RAM. If yours uses (or is configured to use) system RAM that exacerbates these problems.

My guess is that #3 is a likely culprit here. Run Task Manager, click on CPU column, and look for tasks that sit near the top. Of course, SL will always be right near the top. But look for other offenders. Also, after showing the Mem Usage column, click on that and see what floats to the top. (You may have to click these twice to sort biggest first.)

I find the biggest problem is when I'm running SL (or anything, frankly) and my anti-virus disk scan starts to run. Schedule that baby to operate when you won't be using SL. (I sure wish they could make that program less invasive!)

I made a post regarding 'memory leakage' here /142/d1/181730/1.html that may answer some questions.

If you want to find out more about that the SL Client is doing and how it effects your resources, check out the following reporting features built into the client:-
Ctrl+Shift+1, 2, 3, 4, 9

The last one being the most comprehensive. Use the same key combinations to remove those features. I use them a lot to take measurements for Sims whereby the owner has asked our company to install media systems.

I have 2 gigs of ram, when I run the client after about 2 hours the FULL 2 gigs is filled. It starts spilling over into the swap file no doubt, and thats when the issues start. It seems to happen much faster if I TP to new sims often. I have a string of vendors, and when I go to pay leases, I'll often TP into 30-40 sims within the space of an hour. I invariably crash before that hour is up. And yes this is with no other applications open.

Only one third the time to crash. Mighty suspicious if you ask me.

And why does the client start out under 100MB and expand as Torley Linden suggests, to 1.9Gig?

768MB = 5.5 hours.
2Gig = 2 hours.

Hey, pull out some RAM! lol!

I know, dust off the TRS80!

I have the same problem and have a question that hopefully someone can answer for me. I am wanting to get more memory and unsure what kind or is there a better brand that I should buy? I am looking to buy 1gig memory or even more. Oh I have 1 gig right now

Thanks in advance

With system ram, my advice is use matched pairs (same manufacturer, speed etc), depending on the available memory banks (slots) you have on your motherboard. Taking care in the configuration of how you fill those banks and in which order. If you are unsure, either take it to a respected outlet and get them to install it, or remove any existing memory chips and install a large single one on bank 0. Some memory configurations may require updating the system BIOS.