DragonPrime - LoGD Resource Community
Welcome Guest
  • Good morning, Guest.
    Please log in, or register.
  • June 25, 2016, 03:25:43 AM
Home Forums News Links Downloads Login Register Advanced Search
* * *
DragonPrime Menu
Login
 
 
Resource Pages
IRC Channels
Search

Pages: [1] 2   Go Down
  Print  
Author Topic: Increasing the size of your PHP Memory Limit  (Read 10414 times)
0 Members and 1 Guest are viewing this topic.
CortalUX
Dwellings Project
Mod God
*****
Offline Offline

Posts: 796


Wogga! Meep!


View Profile WWW
« on: March 31, 2005, 09:08:16 AM »

(this is meant as a helpful guide)
If you are getting empty pages, or 'this page cannot be displayed' errors, try this and increase the memory limit on your php installation.
There are three routes to do this.

Route 1:
Create a file called 'prefixes.php'.
Type the following within it:
Code:
<?php
ini_set
("memory_limit","16M");
?>
Save the file.
Place the file in the '
lib' directory of your LotGD installation.

Route 2:
Create a file called '.htaccess'.
Type the following within it:
Code:
<FilesMatch "\.(php|html?)$">
php_value memory_limit 16M
</FilesMatch>
Save the file.
Place the file in your main LotGD directory.


Route 3:
If you have access to a file called 'php.ini', open it up.
Look for a value somewhere called
memory_limit
Change the number after it to
16.
If the value isn't in there, add the following:
Code:
memory_limit 16M
Save the file.

If the number 16 doesn't work, you can increase the number. Some users have found 20, 24, 40, and even 64 to help. 16 is recommended for less memory-hungry installations.
« Last Edit: May 30, 2007, 10:09:15 PM by SaucyWench » Logged
gilgalad
Guest
« Reply #1 on: March 31, 2005, 09:34:51 AM »

Im dont have root server so i cant edit my php.ini file. What can I do... My hosting firm said that "we cant upgrade your 8m"  Huh
Logged
CortalUX
Dwellings Project
Mod God
*****
Offline Offline

Posts: 796


Wogga! Meep!


View Profile WWW
« Reply #2 on: March 31, 2005, 10:16:26 AM »

Im dont have root server so i cant edit my php.ini file. What can I do... My hosting firm said that "we cant upgrade your 8m"  Huh
Try the .htaccess or the prefixes.php route.
They may have said that, but it doesn't necessarily mean it won't work... some people talk a load of rubbish.
Logged
Dannic
Guest
« Reply #3 on: March 31, 2005, 06:34:42 PM »

mostly because they are just a reseller and have no clue how to do it.
Logged
gilgalad
Guest
« Reply #4 on: April 04, 2005, 12:15:41 PM »

i m still cant enter the modules.php ...
Logged
Kendaer
Guest
« Reply #5 on: April 07, 2005, 01:55:18 PM »

I've meant to mention this before, but the use of the 'prefixes.php' is not really a good way to do this.

That file was meant to provide Eric and myself with a mechanism for overriding the default prefix on specific tables if (for instance) we wanted to share tables between multiple installs of logd (like for instance the monsters table or the masters table).  It wasn't meant to be used by random devs.

What I am doing however for the next release (which will be 1.0.2) is including a file called lib/local_config.php which is intended for that purpose.

By default this file will contain an ini_set() to up the memory limit to 64 MB.

Unfortunately when you go and try to install humongous amounts of module file (OR go and list a huge number of uninstalled modules!) you end up requiring (and thus including and compiling) all of those modules into your code in one fell swoop.  This causes the memory to baloon and thus causes problems.

One thing I will note is that both Eric and I run the turck MMCache php extension which does caching of compiled PHP.  This is a speedup win, and I firmly believe it helps cut down on the memory use since neither he nor I experience, nor have ever experienced, this problem even with a completely clean install.

Of course,your mileage may vary, and he and I offer no support for installing nor using the MMCache.. You are completely on your own in that regards.

We will try and track down and see if we can lessen the impact of this issue, but at some point there is just nothing we can do.  We might, via sub-requests be able to alleviate the situation for install/module management, but that really only hides the problem.   Eventually, people will install enough modules on a given hook (village or superuser or newday or what have you) that it will cause enough modules to be loaded/compiled as to exceed the memory limits.

Please note, this is NOT a factor of *modules* per-se.  It's a fact of the sheer amount of code!  If the same amount of code were in-line in the core in the way of 0.9.7, you would have *exactly* the same problem in the long run!
Logged
iamsure
Guest
« Reply #6 on: April 10, 2005, 10:48:15 PM »

The reason the 1.0 installer is having issues with memory size (where previous versions didnt) is because the 1.0 installer stores all the initial data in an array.

The array in turn is over 8mb, so when it is brought into memory, before passing to the database, it requires more memory.

Previous versions simply had admins manually dump sql, so memory available in php wasnt an issue.

Instead, the core dev team could choose to use CSV files to store the data, smaller arrays taken in seperately (and in multiple steps), or some other piecemeal approach to data transfer from php to sql.

But in the meantime, the best solution is to simply increase the memory size.
Logged
Kendaer
Guest
« Reply #7 on: April 11, 2005, 08:08:50 AM »

Sorry, iamsure, but you are barking up the wrong tree on this one.

The setup with the sql statements and table descriptor stuff in the installer has been there since mid 0.9.8 series.  Now yes, stuff has been added to it, but very little in general.  It's not the culprit at least not in and of itself.

It's much more likely to be the number of modules being installed into memory and compiled in a shot and we are working on that by splitting out module pieces into seperate files in order to make the amount that needs to be memory resident at any given point in time the smallest bit possible.
Logged
MightyE
Global Moderator
Captain of the Guard
*****
Offline Offline

Posts: 104


Game Creator MIGHTYE


View Profile
« Reply #8 on: April 11, 2005, 08:31:01 AM »

We've done the SQL tables represented as an array for several releases now, I find it unlikely that that's the problem since it wasn't until we put the module manager into the installer that the memory problems started to happen (the only thing that changed in the installer was this management of the modules, and the problem occurs right on this screen).

It's far more likely that the problem is that the size of the PHP binary, plus the size of the compiled code of core code and all the modules together is what makes the game exceed its memory limits on some distributions.  

In order to manage the modules in the installer, we need to inject them all (parse them and put them in the executable code space) so we can call their module info function.  At no point do we ever inject all the modules at the same time except during the install process.  Of course for people performing an upgrade, only those modules that you don't already have installed (even if not activated) will need to be injected, so you can end up not having a problem because of this.

Both JT and I don't run into the memory limits during a clean install on our own dev areas, so it's my current working theory that the PHP binary counts against the memory footprint in this respect, and some people are running in environments with much more compiled into the core PHP binary, increasing its memory footprint over a threshold where the game cannot load all the modules and the PHP binary with out going over the memory limit.

What we're planning on doing is breaking some of the core modules up into separate files, so that when the module file itself (modules/modulename.php) is inserted into the executable code space, it holds a lower footprint.

In PHP, code is not parsed and placed in executable space until logic allows its include/require/include_once/require_once is actually encountered, and it's compiled just in time.

We'll put nub functions in place that include the appropriate code in most functions only once it's actually necessary to execute that code, and hopefully this will keep the overall memory footprint of PHP + compiled code under your memory limits.

JT and I have done a fair amount of investigation on this on our end, and we've discovered that there's no way in PHP to execute code in a sandbox whose memory is freed up after you exit the sandbox.  That means that the approach we're currently taking toward solving this problem may give us quite a bit of mileage, but it is not a permanent fix.  

Until and unless PHP can offer sandbox functionality, I don't know that there is a real actual permanent fix that lets the game handle an unlimited number of modules.  Once a function has been defined in the compiled code space, you cannot remove that function, nor can you redefine it.  Hence you cannot declare a function garbage and free up the memory it uses after it is defined.  That's a very tough realization for us.

I feel like we've currently exceeded the median use of PHP and are discovering the perimeter restrictions of the language, at least when you don't get to control the environment the language runs in (which we don't if we want to make a game that can run in all setups).

Ultimately, depending on how much mileage we get from the "break everything into pieces" approach, it's possible we might have to go with an approach that doesn't require executing any module code, but rather reading specially formatted data, eg out of comments in the module in order to learn about the module (formal name, author, category, requirements, etc). I considered and rejected that approach when we first started the module work, because I thought it was ugly, but I didn't consider this restriction when I made that decision.
Logged
lonnyl
Guest
« Reply #9 on: April 11, 2005, 08:38:03 AM »

Would it not be possible at install to run a script that would read modules in a catagory, store the data in a temp table.... reload, read the next catagory, store the data, and so on....
this way all of the modules would not run at once (although this may increase load time quite a bit)..... Then use the data from the temp table... this way all the modules are not parsed at once?Huh

Just shooting from the hip on that one.....

It may be possible to do this for preferences as well.... as the prefs.php scans all modules for user_ prefs it's size get's rather large and will be one of the first to break in a lower memory setting environment.
« Last Edit: April 11, 2005, 08:39:32 AM by lonnyl » Logged
iamsure
Guest
« Reply #10 on: April 11, 2005, 11:43:04 AM »

We've done the SQL tables represented as an array for several releases now, I find it unlikely that that's the problem since it wasn't until we put the module manager into the installer that the memory problems started to happen (the only thing that changed in the installer was this management of the modules, and the problem occurs right on this screen).
Actually, me and two other admins ran into the problem on step 7 - and it was with a completely stock/fresh 1.01 install.

It would literally just not go any further, unless we added the memory limit increase, and then it worked fine.

Notably, that section is unremarkable (at first I honestly thought it *couldnt* be in that section), with the exception of loading the array.

However, always willing to admit when I am wrong, oddly enough that *completely* reproducible situation (done over a dozen times in two days!), isnt reproducible any more. Now I get through step 7 just fine with no modification to up the memory. Its .. quite puzzling.

I apologize for simplifying a complex problem - I honestly thought I had found an elegant answer, and wanted to help. It seems that I underestimated the difficulty of this challenge. Good luck to you guys on finding it and fixing it. Sad
Logged
Kendaer
Guest
« Reply #11 on: April 11, 2005, 11:57:44 AM »

At no point do we ever inject all the modules at the same time except during the install process.

This is not 100% true Eric.

There is one other case where pretty much all modules get loaded into memory, and that is the handling of the preferences page where any module which defines that it has preferences has to be loaded in order to see if there are any user_* prefs. (we do check to make sure that there is a prefs section before loading the module, but a lot of them have prefs but none which can be set by the general user).   So one possible optimization we could do here would be to split out the user prefs from the normal prefs into a seperate moduleinfo bit..  That would have some beneficial effect.

Similarly, in the module manager, IF we have a large number of modules in a section (such as say the uninstalled section if no modules are installed for som reason) we will once again iterate over the entire selection of modules.
Logged
MightyE
Global Moderator
Captain of the Guard
*****
Offline Offline

Posts: 104


Game Creator MIGHTYE


View Profile
« Reply #12 on: April 12, 2005, 10:25:50 AM »

Actually, me and two other admins ran into the problem on step 7 - and it was with a completely stock/fresh 1.01 install.
[snip]
However, always willing to admit when I am wrong, oddly enough that *completely* reproducible situation (done over a dozen times in two days!), isnt reproducible any more. Now I get through step 7 just fine with no modification to up the memory. Its .. quite puzzling.

Did you ever see step 7 draw?  When you hit submit on step 7, you go right into step 8, and this is where everyone seems to be having the problem, and also where we do all the module includes.

Quote from: lonnyl
Would it not be possible at install to run a script that would read modules in a catagory, store the data in a temp table.... reload, read the next catagory, store the data, and so on....
this way all of the modules would not run at once (although this may increase load time quite a bit)..... Then use the data from the temp table... this way all the modules are not parsed at once?Huh
Quote

Well, it's an approach I've considered.  We can't break it down by category, since the problem occurs in our attempt to read data such as the category, but we could read 50 or so modules at a time, store it in the user's session, and automatically refresh.  This would probably work, but because of the rest of the architecture here, would be fairly complex, and just a bit kludgey.

Quote from: Kendaer
There is one other case where pretty much all modules get loaded into memory, and that is the handling of the preferences page where any module which defines that it has preferences has to be loaded in order to see if there are any user_* prefs.
Quote

Really!?  I didn't realize that, I thought we were tracking which modules had such settings in some other way.  This at least we could move out into, for example, a module setting, eg:
Code:
$meta = array('prefs'=>$moduleinfo['prefs']....);
save_module_setting('__meta__',serialize($meta),$modulename);
Logged
Kendaer
Guest
« Reply #13 on: April 13, 2005, 07:00:43 AM »

Really!?  I didn't realize that, I thought we were tracking which modules had such settings in some other way.  This at least we could move out into, for example, a module setting, eg:
Code:
$meta = array('prefs'=>$moduleinfo['prefs']....);
save_module_setting('__meta__',serialize($meta),$modulename);

It's be easier just to duplicate them in a user_prefs array (they'd still want to show up in prefs, so should remain there).  The number of these (in the core code!!) are relatively small, but I don't know how many there are in various peoples modules.  I'd rather do this by changing the API but would prefer not to break people's modules unless it's a major version (aka API) change..   I leave this up to the community however.  If there are few enough modules that this would effect, I'd be willing to do it as a two-step process.   One step where both API would work and a warning issued for any non-converted module and the second step to deprecate and remove the first API (after a month or so)..  Actually, the more I think about that second option the better I like it.

It looks like about 7 core modules would be effected.

Unless I hear a strenuous object, I plan on doing that over the next day or so.
Logged
Sichae
iMod God
SVN Users
Mod God
*
Offline Offline

Posts: 3458


If ya didn't get it by now... you're hopeless...


View Profile WWW
« Reply #14 on: April 13, 2005, 07:09:20 AM »

I guess the most objection would come about, after seeing how much work would be needed, in order to change over to a new system of things.

If I only need to change a line, big deal... but if I need to come up with a conversion system, to transfer over all the old settings, to the new settings, then eff that. Smiley

Actually... screw that, I won't mind a new system, if it is to be decreasing server load, even just a little.
Logged

If you didn't understand anything in the above post, don't try to attempt anything suggested.

Pages: [1] 2   Go Up
  Print  
 
Jump to:  


*
DragonPrime Notices
Play LoGD on Dragonprime

Support Us
No funds raised yet this year
Your help is greatly appreciated!
Who's Online
36 Guests, 0 Users
DragonPrime LoGD
Recent Topics
Home Forums News Links Downloads Login Register Advanced Search