Dealing with lots of data.

[PhQ]

I am trying to learn how to deal with lots of data… so I have inserted over 4 million rows into a MySQL table with an unique and with random numbers. I want to use php to find out a rank of an unique row based on the number of votes they get.
And when I retrieve the data from the MySQL database I obviously get the ‘error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32 bytes) in …’ on line’ error.

This is the code that I use to get the error:
[php]$sth = $this->db->query("SELECT votes FROM list ");
$this->toplistItems = $sth->fetchAll(PDO::FETCH_ASSOC);[/php]

How can I deal with large amounts of data?
And yes… i know that I can increase the allocated memory… but let’s say that I can’t.

Thank you in advance.

[PhQ]

Also posted on: http://forums.phpfreaks.com/topic/269449-dealing-with-lots-of-data/

[ttbakiatwoam]

Try limiting your SQL query with a WHERE statement to limit the returned value size.

[m11]

Why do you need all 4 million rows in an array? If you iterate the results instead of sending them all to an array with fetchAll it would eliminate the memory allocation error.

[xzax]

What is your database design?
mention all tables and their attributes.

What output you want?

Answer the above questions.

Meanwhile,
Use the MAX function of mysql query.

Use this “SELECT MAX(votes) from list;”

Large amount of data should be stored only in a database, not in a php script. It should display what is necessary only.