Yesterday a rogue file got uploaded to our server which caused all bookmark submissions to fail! For those of you who had selected the 1-day option, you’ve probably found that all submissions failed for bookmarks submitted yesterday. For anyone else, it should be at most just 1 submission.

Usually we filter out duplicate submissions but in light of this glitch we have temporarily removed the duplicate URL check so if you have bookmarks that have a high level of failures then you can delete them and re-submit them today. Apologies for any inconvenience caused!

All of the payments for the accounts that expired on the 28th have now come through so the issue has been resolved for this month. However, we have also been talking to the developers of the membership software who have now been able to identify the problem. Apparently, in their software the subscription period expires on the same calendar day of the month as the previous one. So, as February only had 28 days, the following period expired on the 28th March. This is a bug as of course if a member initially subscribed on the 31st of the month which was the case for many members then Paypal is always going to send payment on the last day of the month which would be today for this month.

I am now working with the developers to ensure a fix so that it does not occur again. Also, Paypal still do sometimes have delays particularly on the last day of the month so we’re also going to ensure that there is a grace period so the subscription does not expire immediately. These measures will ensure continuous service from now on!

I have some more details on the payment problem – it seems that (for once!) Paypal may not be at fault! For those members who took out the subscription on the 31st of the month (which is most members), the billing day should always be the 31st of the month unless there are less than 31 days in that month.

What appears to have happened is that in February the billing cycle ended on the 28th and Paypal sent the payment on the 28th which is fine but in March, Paypal have reverted to the 31st (which is correct) but the membership software we use has expired the subscription on the 28th which is a bug. This makes a lot more sense than Paypal just being slow – that does happen but usually just for a few hours.

We have now submitted a bug report to the developers of the membership software and in the meantime if any members are still unable to login as Pro, I can manually extend the membership correctly. Please contact support if this is the case!

Paypal still have not submitted the payments that were due to renew on the 28th March so I have now filed a complaint. I have also had a modification to our payment system implemented – there is a now a grace period of 24 hours in which your account will remain in Pro status for 24 hours after the billing period expires. Usually when we get this Paypal problem it is resolved within a few hours but obviously something has gone very wrong this month! If you have a Pro account which has been erroneously reverted to free please contact support.

Just a follow up to the last post … the software we use to manage payments is third party (we use aMember) and so unfortunately we don’t control it but today I have put in a support ticket asking if there is a way that some kind of grace period could be implemented so that an account would not actually expire and be downgraded for something like 24 hours after the billing period had elapsed. That should hopefully allow for Paypal delays! I’ll let you know what the status is on that.

Our membership system is directly tied to Paypal. On the renewal date, Paypal is supposed to send the payment before the billing period is over to ensure continuous service but for a few months now we’ve seen a problem where their subscription payments are delayed by a few hours.

What happens with IMAutomator is that our membership software gets to the end of the billing period and marks that account as expired which downgrades the member to Free status. As soon as Paypal send through the payment, the account is restored back to Pro status.

This is extremely frustrating for us and for affected members but please don’t panic – there is nothing wrong with your account and all data remains intact. Please contact Paypal and ask them to send the subscription payment immediately.

I just wanted to write a post about what we’re working on at the moment. After the launch of Release 2 we encountered a couple of problems. One was that the engine could not keep up with the increased workload (which seemed odd as we’d already re-written the engine to execute around 60 times faster than the old one!) and the second was that also due to the increased number of daily submissions, our server was being blocked by many of the sites we were submitting to.

So these problems have obviously taken priority over anything else. The speed issue has been resolved – we found a bottleneck which was not allowing it to run at its full capacity (a few actually!). Many improvements have been made and it now runs exceedingly fast and processes thousands of submissions concurrently.

The second problem has been a little trickier to resolve and is still on-going. We’ve now started to use a variety of proxy servers for our submissions – this spreads the loads across multiple IP’s and there is no real limit to how many proxies we can use so that solves the initial problem. However, proxies are not as reliable as a direct server connection and many submissions fail with a timeout or some other kind of server error caused by the proxy as opposed to the site itself which is what we are now working to resolve.

We already had in place a re-try mechanism; if a submission fails due to a server error it is retried a number of times before a permanent failure is recorded. Some sites are just a little flaky so this tends to sort out temporary glitches. So we’ve now increased the number of retries (which in turn causes the engine to have to do more work which is why we’ve also been working hard to speed it up!) but some sites are still persisting in giving us trouble for a good percentage of submissions which is not what we want.

The goal of IMAutomator is to create backlinks for our members. More sites = more backlinks. Higher success rate = more backlinks. As we’ve only recent launched a new Release we want to work on these core aspects before introducing any new functionality, so we are splitting our time between improving the success rates, cleaning up any bugs we find and adding new sites to the system. We’re not planning to start work on any new features or the new submission tools until we are happy that our existing tools are solid.

What we’ve also noticed is that many of these recorded failures are not actually failures at all! The way we determine success is that we examine the output returned from the submission process for some kind of success identifier. This varies for each site but usually there is either a success message or we see the details of the bookmark created. What is happening since we’ve started using proxies is that sometimes the submission works but we get a timeout whilst it is returning the result – so our success check records a failure when in fact it did go through!

False negatives are better than false positives but of course this is not ideal! Now that we have re-worked our engine to work a little differently, our next approach is to try a new method of determining success that does not use this output. We’ll be working on that next week. If successful, it may mean that not only are we able to more accurately determine success, but we may even be able to start reporting on the actual URL’s of the bookmarks created! That’s the hope anyway…

Our server was down for some period of time last night and this caused our engine to stop running. Unfortunately it did not re-start as it should have so there is a backlog this morning. This should be cleared fairly soon.

Today we are implementing a major change to the submission mechanism which will speed up the engine and also change the underlying structure of how submissions are processed which will then allow us to make further developments – particularly in the area of success checking. We are working towards a solution that will allow us to more accurately report on the status on a submission and possibly to be able to report on the actual URL’s of the bookmarks generated which we know will be extremely useful for those of you using IMAutomator for client-based work in particular.

Today’s change is the first step towards that goal. The site will be in maintenance mode for some time, the engines will be periodically started and stopped causing (deliberate) delays of a few hours to allow us sufficient data for speed testing.

UPDATE: This has now been completed. There were a few additional failures a few hours ago as we ironed out the kinks but these are resolved now. The delay will be caught up in a few hours time.

There are a couple of annoying, but fairly minor, bugs with the secondary submissions feature. Firstly, there is sometimes an error when you try to schedule a batch – this is a timing issue where the report has pulled in a job in error that is actually on the queue ready to be submitted and when the secondary submission is scheduled, a duplicate is encountered and it fails. This is not permanent – usually if you try again in a little while, it should work.

Secondly, there is a problem with the maintenance of the count of submissions still to do and it fails to count down to 0 properly which means that the current batch stays active even when all submissions are complete. The workaround for this is to manually cancel it and then you’ll need the new reporting screen.

This feature really needs an overhaul so we’re going to be re-vamping secondary submissions completely and hopefully fixing all the issues at the same time. However, this is fairly low priority and we do have some more pressing work to do before that so please bear with us!