Proper way to save your place looping an array/db

[digibucc]

I can throw actual code here but it’s all rather long, so i am going to give an example:

[php]<?php
$max = 10; // max runs per exec
$arr = array_fill(0, 50, 0);
$last = file_get_contents(‘last.txt’);
$n = 0;

echo 'left off on '. $last;

while ($n < $max) {
foreach ($arr as $k => $v) {
if ($k > $last) {
// do something with the value ($v)
$n ++;
}
}
}

file_put_contents(last.txt, $k));
?>

[/php] this isn't anything i use, but i do something similar when i want to save where i left off in a program to come back to later. that way if the program crashes or i have to close it, i haven't lost my place. this is normally when working with database records. it makes it run incrementally and as i said save it's place to return to.

it just feels wrong though, and i’m sure it is. especially the refresh. what is the best way to do this?

I would use array_slice instead of if ($k > $last), but that is normally comparing row values from a db, so i can’t just arbitrarily slice that.

[JimL]

I would just do something like this

[php]<?php
$max = 10;
$lastIteration = !empty($_GET[‘iteration’]) ? (int) $_GET[‘iteration’] : 0;

$originalArray = array_fill(0, 50, 0);
$arr = array_slice($originalArray, $lastIteration, $max, true);

echo 'left off on ’ . $lastIteration . ‘
’;

for ($i = 0; $i < $max; $i++) {
foreach ($arr as $k => $v) {
echo 'Key: ’ . $k . ’ value: ’ . $v . ‘
’;
}
}

$nextIteration = $max + $lastIteration;
if (count($originalArray) > $nextIteration) {
header(“Location: /digibucc/?iteration=$nextIteration”);
}

exit;[/php]

edit: forgot the redirection part

[digibucc]

Thank You

in this instance array_slice would work, but real data is not so simple. each entry is an array, and it depends on the value of a key in each array, so i have to do an if to check them.

as for the $_GET, the problem is that requires the browser to stay open, and requires the sscript to keep functioning to pass that variable. those are the two scenarios i want to account for, and other than file_put or a db write i don’t know how to do that.

I do realize that something of a large magnitude should be done via cli and not a browser, but even given that i want a script to go through the data in chunks as to not overload remote servers, and saving where it left off and starting again later are all i can think of.

[JimL]

You can pass variables when using cli as well Obviously not GET/POST, but it’s definitly possible.

[digibucc]

i can pass it between cron runs? so the next one 12 hours from now can pick up where this left off? I thought db/file_put were it

[JimL]

ah, yes if doing it on seperate cron jobs you need an external store of some sort.

[digibucc]

sadface:
thanks though

[JimL]

I would consider something like this:

[php]<?php

$showHelp = (bool) getopt(‘h’);

if ($showHelp) {
print ’
Help

-m maximum increment steps for each run (10)
-f file to store the increment variable in (store)

';
exit;
}

$max = !empty(getopt(‘m’)) ? getopt(‘m’) : 10;
$file = !empty(getopt(‘f’)) ? getopt(‘f’) : ‘store’;

$store = file_get_contents($file);
$lastIteration = !empty($store) ? (int) $store : 0;

$originalArray = array_fill(0, 50, 0);
$arr = array_slice($originalArray, $lastIteration, null, true);

for ($i = 0; $i < $max; $i++) {
foreach ($arr as $k => $v) {
print 'Processing key: ’ . $k . PHP_EOL;
}
}

print 'left off on ’ . $lastIteration . PHP_EOL;

file_put_contents($file, $max + $lastIteration);[/php]

I’m still holding on to the array slice as it removes all previous iterations. Without sending in a slice length you can process however many key/values from the iteration number as you want. But it might still be a no-go depending on the situation

[digibucc]

thanks yeah it has to cycle a 2 level array

[php]
$max = 25;
$rows = array(
‘row1’ = array(‘10’, ‘field 2’, ‘field 3’, ‘etc’
);

$x = 0;
while ($x < $max) {
foreach ($rows as $n=>$row) {
if ($row[0] > file_get_contents[‘maxrow.txt’]) {
// this row is good, use it, and save the number in $row[0] as the starting number for the next run
file_put_contents(‘maxrow.txt’, $row[0]);
}
$x++;
}
}[/php]

the problem is, each row in rows is not necessarily in order, $row[0] is arbitrary when considering the order of the rows in the rows array, but it’s the only field I need to filter by. I do an initial mysql WHERE to not get anything to low, but i still need to process the returning results with an if because without being in order i can’t count on array_slice.

right?