Truncated Posts

andwhyisit · Sep 6, 2015

A number of posts after the move were truncated (cut short). It doesn't seem to be related to the length of the posts, and it isn't all that common, but I would like to fix whatever posts I can.

Given that I at present have no real way to find out which posts were broken, and given that I can only fix posts on a case by case basis, I will need everyone to help me out by reporting broken posts.

duncathan · Sep 6, 2015

Look through Quick Modding/Hacking Answers. I think it may be related to the < character, which is why so many get cut short in that thread. Do you think that maybe it's trying to parse that as HTML or something stupid like that?

HaydenStudios · Sep 6, 2015

DoubleThink · Sep 7, 2015

It wouldn't surprise me if that were the case, IP Board would display posts after the < if submitted initially but if you went to edit them all the text after would be lost, so it's possible they weren't read properly during the shift. In this case there may be a lot of old posts in the modding section that need fixing D:

andwhyisit · Sep 7, 2015

SELECT * FROM REDACTED WHERE REDACTED LIKE '%<%'
6217 results.
"F**K!"

Noxid · Sep 7, 2015

we should probably just erase those posts entirely, because to leave them half-full is very unprofessional
and of course we simply don't have the man-hours to pull off a replacement like that.

DoubleThink · Sep 7, 2015

What no that's a terrible idea, why would we ever want to lose that much information/history?

Assuming it is the problem (seems increasibly probable), would a script work at all? If not, it's probably only a few hundred posts in actuality, because some will be TSC dumps with multiple commands. I could fix them if nobody else wants to :v

GIRakaCHEEZER · Sep 7, 2015

DoubleThink said:
Assuming it is the problem (seems increasibly probable), would a script work at all? If not, it's probably only a few hundred posts in actuality, because some will be TSC dumps with multiple commands. I could fix them if nobody else wants to :v

I'm not sure if you're aware how SQL works, but it's likely that andy was querying the FORUM_POSTS table or something and checking the POST_CONTENT field. That is, that query isn't counting the number of "<"s in posts but the number of posts with "<" in them. So there's likely 6000+ posts that need fixing.

andwhyisit · Sep 7, 2015

I have a converter from that "BBCode in IPB4" project I abandoned before the IPB to XF move. I can in theory use it to write a script that may auto-fix five-sixths of those posts. So please refrain from deleting posts.

And GIR is correct. If the "posts with < in them were truncated" theory holds up then we have over 6000 broken posts. It isn't just tsc commands but also stuff like :-< and <3 and >_<.

EDIT: I'm working out any kinks in the system before I do an auto-run of the script, but it's looking good. The worst that could happen is that some of these posts may become slightly visually broken but at least they won't be missing anything.

EDIT2: My script is finished and has finished running (it sits at around 1000 lines of code). All but 149 of the broken list pile have been fixed. The remaining posts I'll have to fix manually.

EDIT3: Down to 105 posts left to manually fix.

EDIT4: 75 left to go.

EDIT5: Finished. All 6217 posts should be fixed.