Further discussion on automatic error reporting

[Megiddo]

This is a continuation of the discussion here so as not to overwhelm the thread's original intent with this side-talk.

So, to reiterate, the basic idea is that every time our code has an error we use http.Get() to push data about the bug to our server (IE: http://ulyssesmod.net/bug_report.php?error=myShirtIsOnFireHELP&file=shirt.lua&line=41).

Pros, cons, and mitigation

Pros:

• Automatic error collection to make it easier for us to have as bug-free software as humanly possible.
• We don't have to ask users for their errors. If they come to the forums asking for help and give their server IP we can look up the error they were having (assuming, of course, the presence of an error and not just user stupidity).
• The aggregate data will give us a good idea of priorities... what needs to be fixed first, what can be put off a little while.

Cons:

• Possible spam. In my mind, not a big deal because we'll be able to tell almost at a glance what's real and what's not. Mitigation for this is IP blocking.
• User privacy concerns. Given the nature of what we're dealing with, I highly doubt anyone would even care. Mitigation for this would be to implement an 'error queue' and pop up a menu to a superadmin next time one joins showing all errors in the queue and asking if it's alright to send them off (and showing them exactly what's sent). As far as what data to send, I think the IP (collected from PHP), stacktrace, and version is all we need. I've included a sample stacktrace below for reference.
• We might get reports from people who have modified the files and introduced an error. Just like spam, this would be another one that would be easy to detect when compared against the stack trace. But, I think it's worth having these anyways because it might reveal a use for one of our functions we never thought of before, or a logic shift, whatever.

lua5: stacktrace.lua:10: the error
stack traceback:
        [C]: in function `error'
        stacktrace.lua:2: in function `p3'
        stacktrace.lua:6: in function `p2'
        stacktrace.lua:10: in function `p1'
        stacktrace.lua:15: in function `luaf'
        stacktrace.lua:19: in main chunk
        [C]: ?

[MrPresident]

Personally I really like this idea. I don't know how much my opinion weighs as I'm not a developer of ULX, but I think that having more technical information about bugs will really help you more than just someone telling you it doesn't work on the forums.

Also, this could also be a great way for you to unveil bugs and errors that lie dormant that the general user may never know are even there.

[MrPresident]

Okay so I just had a brain picture thing...


This relies heavily on the ability of lua and php to hash a string.. I dont know if either language has the ability... but this could be a way to stop or severely limit the spam that this could cause.

lua hashes the servers IP. Can you get a users IP with lua? then it sends it with the error report. The php on the server end hashes the IP of the client sending it. If the 2 strings don't match.. then it disregards the report.


Joe Shmoe figures out what is going on blah.com/report.php?error=HAHAHA GUYS SUCK&file=suck.it&c=99BC5D3848AE

^^ or whatever.. but little does he know that his IP is not the same as the server so the Hash he used which he didn't know was a hash doesn't pan out and the server disregards.


I don't know.. just brainstorming. Sorry for my chicken scrabble.

[Megiddo]

A good idea, but I really don't think spam will be much of a problem. I doubt anyone would bother to begin with, but if they do, it would be pretty easy to filter it out.

[JamminR]

Re: File editing - How difficult would it be to store a hash (md5?) of our various files, then have the bug report URL contain it regarding the stack trace. This would quickly show file modification.

Mr.President, opinions are valued here, especially in your case because there is intelligent thought behind them.

Right Meg? My sometimes opposing opinion has led to better things for ULib/ULX at times (for life of me I can't remember link examples)
I just remember one time Megiddo and I having (friendly) debate over something, him coming back next day with totally new different idea that worked better than what either of us had discussed.

It's the similar opinions you really have to watch out for.
Brain cell light bulbs have been known to light up the night for hours at a time without needing a recharge.

Stick! Mmm. Cookies!

[Megiddo]

I don't ever really mean to put other people's ideas down, sorry if y'all felt like I was minimizing Mr. P's opinion. It's just I honestly don't expect people to abuse the system, and if they do it's something we can take care of down the road.

Also JamminR, how about how to handle immunity in ULib SVN? Neither of us was happy with the other's idea, but then we came up with something completely new.

[JamminR]

Yeah! That's just one of many times we've (all of us) worked well as a team. Discussion!
I think that's Garry's problem. He's a one man island when it comes to Gmod, and it often shows.

(Oh, and I didn't see your reply as putting Mr. President down)

[Stickly Man!]

Re: Privacy concerns

Perhaps we should make it an optional feature (users must enable it via ULX settings) by enabling it, specifying their forum username and password (to prevent unauthorized .. error.. reporting). Perhaps instead of password we could make the users set a passphrase on the forum, then use that instead of their forum password.

Then, no IP will need to be stored-- we can look up errors based on their username. Also, that being said, in case we need their IP for joining their server and checking around, we can specify an optional server IP address field for user accounts, which (should?) only be visible by admins.

[Megiddo]

Re: Privacy concerns

Perhaps we should make it an optional feature (users must enable it via ULX settings) by enabling it, specifying their forum username and password (to prevent unauthorized .. error.. reporting). Perhaps instead of password we could make the users set a passphrase on the forum, then use that instead of their forum password.

Then, no IP will need to be stored-- we can look up errors based on their username. Also, that being said, in case we need their IP for joining their server and checking around, we can specify an optional server IP address field for user accounts, which (should?) only be visible by admins.

Excellent idea! I really like it and think it's something worth doing... even if we'll only have a few users doing it.