weird bug on close / reopen

[ofZach]

this is a little strange, I'm still debugging it but thought I'd mention it here. if you load an image after arCore.setup() arcore seems to freeze when you close and reopen the app. You don't even need to draw the image, just load it after arcore.setup(). If you load it before setup it's ok.

I think it's texture related, this is what I see in the console :

W/GLConsumer: [SurfaceTexture-1-32421-1] bindTextureImage: clearing GL error: 0x502
E/Adreno-UNKNOWN: glEGLImageTargetTexture2DOES:468: GL_INVALID_OPERATION
glEGLImageTargetTexture2DOES:468: GL_INVALID_OPERATION
E/GLConsumer: [SurfaceTexture-1-32421-1] bindTextureImage: error binding external image: 0x502
E/native: session.cc:493 generic::failed_precondition: VIO is not tracking.

perhaps it has to do with the order where objects are fixing their texture reloading? or something about OES vs there texture types. I'll poke around some more. I should add I'm using @boehm-e's branch so I'm not 100% sure this effects all version but I'll try to test. To recreate, simply load an image in setup() of the exampleBasic

void ofApp::setup(){
    arcore.setup();
    initialized = false;
    img.load("a.png");
    ofBackground(0);
    ofSetOrientation(OF_ORIENTATION_DEFAULT, false);
}

[ofZach]

a potential fix is to call reallocate the OES texture and pass it on on resume:

void ofxARCore::reloadTexture() {
    GLuint texId = setupTexture();
    JNIEnv *env = ofGetJNIEnv();
    jmethodID javaSetupMethod = env->GetMethodID(javaClass, "setup2", "(I)V");
    if (!javaSetupMethod) {
        ofLogError("ofxARCore") << "setup(): couldn't get java setup for ofxARCore";
        return;
    }
    env->CallVoidMethod(javaTango, javaSetupMethod, texId);
}


GLuint ofxARCore::setupTexture(){

    GLuint texId[1];
    glGenTextures(1, texId);

    glEnable(GL_TEXTURE_EXTERNAL_OES);
    glBindTexture(GL_TEXTURE_EXTERNAL_OES, texId[0]);

    glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
    if (!ofIsGLProgrammableRenderer()) {
        glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
    }

    glDisable(GL_TEXTURE_EXTERNAL_OES);

    texture.clear();
    // Set the externally created texture reference
    texture.setUseExternalTextureID(texId[0]);
    texture.texData.width = 1;
    texture.texData.height = 1;
    texture.texData.tex_w = 1;
    texture.texData.tex_h = 1;
    texture.texData.tex_t = 1; // Hack!
    texture.texData.tex_u = 1;
    texture.texData.textureTarget = GL_TEXTURE_EXTERNAL_OES;
    texture.texData.glInternalFormat = GL_RGBA;
    return texId[0];
}


void ofxARCore::pauseApp(){
    JNIEnv *env = ofGetJNIEnv();
    env->CallVoidMethod(javaTango, env->GetMethodID(javaClass, "appPause", "()V"));
}

void ofxARCore::resumeApp(){
    reloadTexture();
    JNIEnv *env = ofGetJNIEnv();
    env->CallVoidMethod(javaTango, env->GetMethodID(javaClass, "appResume", "()V"));


    // Check if permissions have been granted, and the session isn't setup yet
    if(!_sessionInitialized){
        if(ofxAndroidCheckPermission(OFX_ANDROID_PERMISSION_CAMERA)) {
            setupSession();
        }
    }
}

and in ofxArCoreLib java

	public void setup2(int texId){    // this could have a better name
		mTexId = texId;
		updateTexture();
	}

I'm not confidant in android texture / reload wonkiness to know if this is the ideal fix to the problem but it seems to help!

[boehm-e]

@ofZach : first thank you for inspiring a lot of people like me who did not know anything about creative coding :)

I have that issue too (but did not investigate), so I tried your possible fix.
Now onResume, I get it not crashing ( that's better than before ;) ), so I added your suggestion to this commit : link

But : now when I resume my app, the textures that i'm drawing does seems to exist (it do not crash when i draw them), but i cannot see them :/
Have you the same issue ?

Thank you

[ofZach]

thanks! I don't have that issue with the sample I posted above (just loading an image before or after arCore.setup()) --- are you talking about the texture from arCore or something else? In my larger project I am working on I do reallocate my own FBOs that I am using internally in my app but images reload automatically. I'm not 100% sure about how OF handles the reloading of textures on pause / resume but it feels like some things (ofImage) work automatically but fbos you need to handle yourself.

[boehm-e]

Yes sorry I was talking about the textures that I create (not the texture of the arcore session).
Ok, I am using FBOs, so it might be the issue. Will try with ofImage to better understand what's wrong.

[boehm-e]

👍 It does comes from FBOs : when using ofImage (and your fix) I am able to resume my app and use it as normal.

Can I know how you reallocate your FBOs ? ( I'm kind of new to textures, and that sort of things )

[ofZach]

yeah in my experience I have to reallocate fbos on my own but ofImage and other image based objects in OF like font handle reload on their own. Again, it's a little unclear how reloading works on android -- cc @arturoc who might have some insight...

in terms of reallocated I just do

fbo.clear();
fbo.allocate()

in ofApp::reloadTextures or in the case of this app I've found I need to set a flag there and then in update() reallocate the fbos I am using ...

[boehm-e]

Thank you that's very clear.

[arturoc]

@ofZach that's strange ofFbo should reallocate itself after the context is lost. When allocated (or copied...) it registers itself to the reloadGL event in android: https://github.com/openframeworks/openFrameworks/blob/master/libs/openFrameworks/gl/ofFbo.cpp#L712 when that event is triggered it calls reloadFbo which calls allocate using the initial settings: https://github.com/openframeworks/openFrameworks/blob/master/libs/openFrameworks/gl/ofFbo.cpp#L719

if that's not working please open an issue in the main repo

[ofZach]

@arturoc as sorry that might be my misunderstanding -- I think my fbos are empty and I need to redraw them but I thought I needed to reallocate them. I may be doing that unnecessarily -- let me check !

[boehm-e]

I have two build of my app:
In the first I use ofImage, and when i resume the app, I still can see them.
In the second, I use ofFbo and when i resume, I cannot se them (they exist, bcs it do not crash, but they do not display)

I thought, it was, as ofZach said, realted the fact that i do not reallocate my FBOs.
So i'm kind of lost here ;)

Will investigate

[arturoc]

The fbos don't need to be reallocated but the content of the textures they hold is lost.

When using an ofImage we have a copy of their content in the cpu but when using a texture there's only the copy in the gpu so when the gl context is lost the content of the texture is lost and we don't have a way to recover it.

You need to recreate the content but not reallocate the fbo as zach was pointing out

[boehm-e]

@arturoc Thank you
But what if the content is created dynamically ?
There is no way to keep it when app resume ?

[arturoc]

yes but you should take care of doing so, the easiest is usually to redraw it but you can also download the contents of the texture to an ofPixels when the context is about to be lost and uploading it again when it's reloaded if you don't know how to redraw it from scratch after the context is lost