sclang interpret string size limit

[ifranco]

Hello,
I've been using supercollider.js to build a new web editor for Prynth. I've implemented block evaluation, like in SC IDE, via a call of the .interpret method.
Apparently there is a limit to the size of the string that can be passed to interpret...
Can you confirm this?
If this is a reserved size for the string, can it be increased in order to evaluate bigger chunks of code?

Thanks,
Ivan

[crucialfelix]

Of the top of my head I think sclang has a limitation. But that's like 32k

How big is your string ?

[ifranco]

Hi Chris, The string is 9250 characters. The callback error I get is: {"data":{},"type":"SyntaxError","error":{"msg":null,"file":null,"line":null,"charPos":null,"code":"”}} It seems the string is not simply not passed. The same exact code works fine when passed via POST to a function, that saves to a file file and runs it via .executeFile. It makes me think that a string limitation is not on the HTTP method. Thanks, Ivan

…

On Aug 24, 2017, at 5:28 AM, Chris Sattinger ***@***.***> wrote: Of the top of my head I think sclang has a limitation. How big is your string ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[crucialfelix]

That's not the problem. You can send quite large text. Could you post just the part of your code where you call interpret ? thanks

…

On Thu, Aug 24, 2017 at 4:15 PM ifranco ***@***.***> wrote: Hi Chris, The string is 9250 characters. The callback error I get is: {"data":{},"type":"SyntaxError","error":{"msg":null,"file":null,"line":null,"charPos":null,"code":"”}} It seems the string is not simply not passed. The same exact code works fine when passed via POST to a function, that saves to a file file and runs it via .executeFile. It makes me think that a string limitation is not on the HTTP method. Thanks, Ivan > On Aug 24, 2017, at 5:28 AM, Chris Sattinger ***@***.***> wrote: > > Of the top of my head I think sclang has a limitation. How big is your string ? > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub, or mute the thread. > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#33 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANWcoS3CT18gvJWxSdNMjRAtesN-wD_ks5sbYVrgaJpZM4PATM8> .

[ifranco]

I realized this problem via complex SC code, but then did a simple test with several lines of:

…

--- fork{ "working...".postln; (0.1).wait; "working...".postln; (0.1).wait; ... "done".postln; } — Let say 300 lines of it should be enough. Ivan

On Aug 24, 2017, at 10:35 AM, Chris Sattinger ***@***.***> wrote: That's not the problem. You can send quite large text. Could you post just the part of your code where you call interpret ? thanks On Thu, Aug 24, 2017 at 4:15 PM ifranco ***@***.***> wrote: > Hi Chris, > The string is 9250 characters. > The callback error I get is: > > > {"data":{},"type":"SyntaxError","error":{"msg":null,"file":null,"line":null,"charPos":null,"code":"”}} > > It seems the string is not simply not passed. > > The same exact code works fine when passed via POST to a function, that > saves to a file file and runs it via .executeFile. > It makes me think that a string limitation is not on the HTTP method. > > Thanks, > Ivan > > > On Aug 24, 2017, at 5:28 AM, Chris Sattinger ***@***.***> > wrote: > > > > Of the top of my head I think sclang has a limitation. How big is your > string ? > > > > — > > You are receiving this because you authored the thread. > > Reply to this email directly, view it on GitHub, or mute the thread. > > > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#33 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AANWcoS3CT18gvJWxSdNMjRAtesN-wD_ks5sbYVrgaJpZM4PATM8> > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[crucialfelix]

Let me try it. Sorry I've been quite busy this last week.

[ifranco]

Hi Chris,
have you had an opportunity to look at it? Have you been able to reproduce the same error?

Thanks,
Ivan

[crucialfelix]

I am trying with strings as large as 221k and it is still correctly returning a SyntaxError for me.

Can you post the code or at least something that shows what it is ? a gist or pastebin would be fine. There is some syntax error in there but I don't know why it returns nothing in the code slot of the error.

I do notice that you have a printer quote in there "code":"” which is weird

[woolgathering]

I can't help but think that this and crucialfelix/atom-supercollider#71 are related. I ran into the issue again yesterday and have tried to find what might be happening. I'm getting the exact same callback error as @ifranco so I followed it down.

The string that is meant for interpreting is fine and complete up through the call to SuperColliderJS.interpret in the write function in sclang.js (src/lang/sclang.js, line 379 and its analogue in lib/lang/sclang.js in the atom-supercollider package). Inside SuperColliderJS, the string is truncated to the spot that throws the error, for me, at 8189 characters every time. So something is happening with the call to SuperColliderJS.interpret that is truncating the string.

After reading a bit more, I have a hunch that it might have to do with a UDP data length limit of 65,507 bytes (8188 UTF-8 characters... Coincidence?). But I don't know why it would work in the SC-IDE or enough about UDP to go any further. To be honest, I'm pretty sure atom-supercollider is using UDP to talk to sclang but I'm not even 100% certain.

Correct me if I'm totally off base. I'd really like to help get this going since it's a serious obstacle in my workflow.

[crucialfelix]

It's not using a socket (UDP or TCP) at all. It is connected directly to the standard out / standard in of the sclang process. So it is in all likelihood a problem with sclang posting too much stuff and messing it up. I've seen that happen. JMC decided the music was the important thing and if the post buffer is too big or filling too fast then it just doesn't bother to print it all.

There is a PR here: #37 that DOES solve it. It uses a TCP socket. I want to test and merge it, but there are issues to solve with running multiple sclang (which I have done often). But it should be a more stable communication channel.

[ifranco]

Hi Chris, I'm really sorry that I still haven't followed up on this, although it never left my todo list. I just wanted to mention that the problem I encountered is not with posting, but simply that it seems that the evaluated code is simply not passed to sclang. Because there is no execution, there are also no errors. Thanks, Ivan

…

On Oct 30, 2017, at 5:59 PM, Chris Sattinger ***@***.***> wrote: It's not using a socket (UDP or TCP) at all. It is connected directly to the standard out / standard in of the running process. So it is in all likelihood a problem with sclang posting too much stuff and messing it up. I've seen that happen. JMC decided the music was the important thing and if the post buffer is too big or filling too fast then it just doesn't bother to print it all. There is a PR here: #37 that DOES solve it. It uses a TCP socket. I want to test and merge it, but there are issues to solve with running multiple sclang (which I have done often). But it should be a more stable communication channel. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[woolgathering]

Yes, my problem seems identical to @ifranco; i.e. it's not a posting problem as there is no execution. I tested the string meant for interpreting by writing it to a file in SuperColliderJS.interpret. Immediately before the method is called, it writes in its entirety but within the *.interpret method, it is truncated.