I am writing a script that follows links from a page and need to keep track of the links that have already been visited so i dont visit them again. I tried using a global array but am having trouble using my recursive method. For example:
http://example.com/foo will have links to
and these links may themselves contain http://example.com/foo as a link which i dont want to follow as it has already been traversed.
Any ideas would be greatly appreciated.