Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] GST - GST_OBJECT and GST_MESSAGE_SRC #197

Open
mantielero opened this issue Jan 11, 2022 · 7 comments
Open

[Question] GST - GST_OBJECT and GST_MESSAGE_SRC #197

mantielero opened this issue Jan 11, 2022 · 7 comments

Comments

@mantielero
Copy link

I am trying to translate the following line from basic-tutorial-3.c:

/* We are only interested in state-changed messages from the pipeline */
          if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data.pipeline)) {

GST_MESSAGE_SRC

As indicated here, my understanding is that:

GST_MESSAGE_SRC (msg)

would be equivalent to:

msg.impl.src

when var msg:Message. And I understand that typeof(msg.impl.src) == ptr Object00.

GST_OBJECT

This is less clear to me. It is defined in gstobject.h, but it is not explained in the macros definitions.

What would be the equivalent to GST_OBJECT in gst.nim?

@StefanSalewski
Copy link
Owner

I think your understanding is correct. Actually you know much more about gst than I do, as I know nothing. We only have the gst libs in gintro, because someone asked for it long time ago, and I have not refused to add it. Learning gst would be a really big task for me, without any real benefit. And I have really a lot of other task to do, I would like to finish the Nim book this year, work on my CAD tool, maybe then write some more sections of the GTK book. Rewrites of gintro is also on the list, but I have still some hope that someone other may do that.

For your questions, I had to ask Google and grep:

if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data.pipeline)) {

https://gstreamer.freedesktop.org/documentation/gstreamer/gstmessage.html?gi-language=c

#define GST_MESSAGE_SRC(message) (GST_MESSAGE_CAST(message)->src)

So GST_MESSAGE_SRC is a C macro, which generally is not directly supported by gobject-introspection. Sometimes there are also conversion functions available, even when in the C examples the macros are used. A plain Nim cast should work, for a clean solution one may have to add converter functions -- note I am not talking about the Nim converters here.

https://gstreamer.freedesktop.org/documentation/gstreamer/gstobject.html?gi-language=c

GstObject provides a root for the object hierarchy tree filed in by the GStreamer library. It is currently a thin >wrapper on top of GInitiallyUnowned. It is an abstract class that is not very usable on its own.

grep  -B2 " Object00\*" ~/.nimble/pkgs/gintro-0.9.6/gintro/gst.nim 
type
  Object* = ref object of gobject.InitiallyUnowned
  Object00* = object of gobject.InitiallyUnowned00

So Nims GstObject type is gst.Object and gst.Object00 for the low level one.

That is all what I can tell you for now, I think you know it already. I have not much hope that someone other will see this open issue and will be willing to help you unfortunately.

@StefanSalewski
Copy link
Owner

I forgot to mention one important point:

As gst is such a low level library, maybe low level bindings would work better for you: Mr. Munch recently created a wrapper generator called fluthark, and there are still the tools of Mr Shashlik, I think called nimterop and toast.

@mantielero
Copy link
Author

Thanks for your comments.

Offtopic: maybe some elements from the following are useful to you
https://github.com/mantielero/solvespace.nim
https://github.com/mantielero/occt.nim

@StefanSalewski
Copy link
Owner

I noted that your previous question was also about basic-tutorial-3.c, so I had a look at that file to see if it is really that demanding. Unfortunately the C code does not work at all for me with wayland, see

https://discourse.gnome.org/t/gstreamer-tutorials-and-wayland/8631

Well, I may see how far I can convert it to Nim for you, looks not that difficult at first.

@mantielero
Copy link
Author

That would be great. I was also having issues with the callback, but I hope that each step will be easier.

@StefanSalewski
Copy link
Owner

That would be great. I was also having issues with the callback, but I hope that each step will be easier.

OK, here is a compiling and working version -- for me currently not with wayland, but that is a different problem. I did the conversion fully manually, as I had the feeling that c2nim would not help much in this case. As I know nothing about gst I had to use Google and grep a lot. As you already found out, accessing the msg src seems to be a problem, I commented that section out. And the gerror parsing seems to be wrong, I commented that out too. That function seems to be fixable, but we have to investigate why the automatically generated code is wrong. But it is ugly, getting a gerror as result, which we have to free manually, is ugly. With msg src, we would have to think carefully how a really clean fix may look. Your initial suggestion, using the impl field, the 00 data type and the unexported function should work, but is ugly. WE can always generate a new proxy object, but we have always to consider object life times and and correct memory deallocation. All that needs some carefully thinking.

Well, you see that using gintro is possible even for gst. Took me 3 hours, from C to Nim, which is not that bad regarding the fact that I know nothing about gst.

I think all the further tuning you should be able to do yourself -- when you are really interested in this topic.

$ lt
total 8076
-rwxr-xr-x 1 salewski salewski  196248 Jan 13 23:10 basic_tutorial_3
-rw-r--r-- 1 salewski salewski    6649 Jan 13 23:10 basic_tutorial_3.nim
-rw-r--r-- 1 salewski salewski    6651 Jan 13 23:03 basic_tutorial_3.nim.bak
-rw-r--r-- 1 salewski salewski    5537 Jan 13 17:50 basic-tutorial-3.nim.bak
-rw-r--r-- 1 salewski salewski 7798316 Jan 13 13:01 basic-tutorial-1.c_debug.txt
-rwxr-xr-x 1 salewski salewski   20440 Jan 13 12:42 basic-tutorial-3
-rw-r--r-- 1 salewski salewski    5232 Jan 13 12:40 basic-tutorial-3.c
-rwxr-xr-x 1 salewski salewski  151608 Jan 13 11:10 gstBasicTutorial1
-rw-r--r-- 1 salewski salewski     753 Jan 13 11:10 gstBasicTutorial1.nim
-rwxr-xr-x 1 salewski salewski   19168 Jan 13 11:05 basic-tutorial-1
-rw-r--r-- 1 salewski salewski     818 Jan 13 11:05 basic-tutorial-1.c
-rw-r--r-- 1 salewski salewski    5394 Jan 13 10:42 basic-tutorial-3.c.bak
-rwxr-xr-x 1 salewski salewski   20432 Jan 13 10:35 t3
-rw-r--r-- 1 salewski salewski    5280 Jan 13 10:34 t3.c
salewski@nuc ~/mantelhere $ nim c basic_tutorial_3.nim
Hint: used config file '/home/salewski/Nim/config/nim.cfg' [Conf]
Hint: used config file '/home/salewski/Nim/config/config.nims' [Conf]
.........................................................................................
Hint:  [Link]
Hint: gc: refc; opt: none (DEBUG BUILD, `-d:release` generates faster code)
82040 lines; 1.142s; 116.844MiB peakmem; proj: /home/salewski/mantelhere/basic_tutorial_3.nim; out: /home/salewski/mantelhere/basic_tutorial_3 [SuccessX]
salewski@nuc ~/mantelhere $ ./basic_tutorial_3 
Received new pad src_0 from source:
It has type video/x-raw which is not raw audio. Ignoring.
Received new pad src_1 from source:
Link succeeded (type audio/x-raw).
Debugging information: ../gstreamer-1.18.4/plugins/elements/gstqueue.c(990): gst_queue_handle_sink_event (): /GstPipeline:test-pipeline/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstQueue:vaapi-queue:
streaming stopped, reason not-linked (-1)
# https://gitlab.freedesktop.org/gstreamer/gst-docs/blob/d2469972f06bc2ef2936fd8ab4b708a862f3d220/examples/tutorials/basic-tutorial-3.c

import gintro/gst
#from ginto/glib import str_has_prefix
from strutils import startsWith, `%`
import gintro/[glib, gobject, dummygtk] # we need gtk or gtk4 for connect(), maybe dummygtk will do

proc g_printerr(s: string) = stderr.write(s)

proc g_print(s: string) = stdout.write(s)

proc toStringVal(s: string): Value =
  let gtype = gStringGetType() # typeFromName("gchararray")
  discard init(result, gtype)
  setString(result, s)

# /* Structure to contain all our information, so we can pass it to callbacks */

type
  CustomData = ref object
    pipeline: gst.Element
    source: gst.Element
    convert: gst.Element
    sink: gst.Element

# /* Handler for the pad-added signal */
# /* This function will be called by the pad-added signal */
proc pad_added_handler(src: gst.Element; new_pad: gst.Pad; data: CustomData) =
  var
    sink_pad: gst.Pad  = gst.get_static_pad(data.convert, "sink")
    ret: gst.PadLinkReturn
    new_pad_caps: gst.Caps
    new_pad_struct: gst.Structure
    new_pad_type: string

  g_print ("Received new pad $1 from $2:\n" % [getName(new_pad), getName(src)])

  block gotoExit:
    # /* If our converter is already linked, we have nothing to do here */
    if (gst.is_linked(sink_pad)):
      g_print("We are already linked. Ignoring.\n")
      break gotoExit

    # /* Check the new pad's type */
    new_pad_caps = gst.get_current_caps(new_pad)
    new_pad_struct = gst.get_structure(new_pad_caps, 0)
    new_pad_type = gst.get_name(new_pad_struct)
    #if (not glib.str_has_prefix(new_pad_type, "audio/x-raw")):
    if (not startsWith(new_pad_type, "audio/x-raw")):
      g_print ("It has type $1 which is not raw audio. Ignoring.\n" % [new_pad_type])
      break gotoExit

    # /* Attempt the link */
    ret = gst.link(new_pad, sink_pad)
    # #define GST_PAD_LINK_FAILED(ret) ((ret) < GST_PAD_LINK_OK)
    # if GST_PAD_LINK_FAILED (ret):
    if ret < gst.PadLinkReturn.ok:
      g_print ("Type is $1 but link failed.\n" % [new_pad_type])
    else:
      g_print ("Link succeeded (type $1).\n" % [new_pad_type])

#exit:
  # /* Unreference the new pad's caps, if we got them */
  if new_pad_caps != nil:
    discard # is here some work to do?
    # gst.caps_unref(new_pad_caps)

  # /* Unreference the sink pad */
  # gst.object_unref(sink_pad)

proc main: int =
  var
    data: CustomData
    bus: gst.Bus
    msg: gst.Message
    ret: gst.StateChangeReturn
    terminate: bool

  # /* Initialize GStreamer */
  gst.init() # gst_init (&argc, &argv);

  # /* Create the elements */
  new data
  data.source = gst.make("uridecodebin", "source") # gst_element_factory_make
  data.convert = gst.make("audioconvert", "convert")
  data.sink = gst.make("autoaudiosink", "sink")

  # /* Create the empty pipeline */
  data.pipeline = gst.newPipeline("test-pipeline")

  if (data.pipeline == nil or data.source == nil or data.convert == nil or data.sink == nil):
    g_printerr("Not all elements could be created.\n")
    return -1

  # /* Build the pipeline. Note that we are NOT linking the source at this
  #  * point. We will do it later. */
  # gst.bin_add_many(GST_BIN(data.pipeline), data.source, data.convert, data.sink, NULL)
  discard gst.add(gst.Bin(data.pipeline), data.source)
  discard gst.add(gst.Bin(data.pipeline), data.convert)
  discard gst.add(gst.Bin(data.pipeline), data.sink)

  if not gst.link(data.convert, data.sink):
    g_printerr("Elements could not be linked.\n")
    # gst_object_unref (data.pipeline);
    return -1

  # /* Set the URI to play */
  setProperty(data.source, "uri", toStringVal("https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm"))
  #g_object_set (data.source, "uri",
  #    "https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm",
  #    NULL);

  # /* Connect to the pad-added signal */
  connect(data.source, "pad-added", pad_added_handler, data)

  # /* Start playing */
  ret = gst.set_state(data.pipeline, gst.State.playing)
  if ret == StateChangeReturn.failure: # GST_STATE_CHANGE_FAILURE:
    g_printerr("Unable to set the pipeline to the playing state.\n")
    #gst_object_unref (data.pipeline);
    return -1

  # /* Listen to the bus */
  bus = gst.get_bus(data.pipeline)
  while true:
    # msg = gst.timed_pop_filtered(bus, gst.CLOCK_TIME_NONE, GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_ERROR | GST_MESSAGE_EOS)
    msg = gst.timed_pop_filtered(bus, gst.CLOCK_TIME_NONE, {MessageFlag.stateChanged, MessageFlag.error, MessageFlag.eos})

    # /* Parse message */
    if (msg != nil):
      var err: ptr glib.Error
      var debug_info: string
      # define GST_MESSAGE_TYPE(message)       (GST_MESSAGE_CAST(message)->type)
      # case GST_MESSAGE_TYPE (msg):
      var  mt: gst.MessageType = msg.getType # a set[enum], casting to integer may be dangerous?
      #case msg.getType
      if gst.MessageFlag.error in mt:
        # of gst.MessageFlag.error.ord: # GST_MESSAGE_ERROR:
        gst.parse_error(msg, err, debug_info) # caution, this is wrong, cstring is not freed. Really hard and ugly!
        ###g_printerr("Error received from element $1: $2\n" % [getName(msg.src), err.message])
        g_printerr("Debugging information: $1\n" % [if debug_info.len > 0: debug_info else: "none"])
        clear_error(addr err)
        #g_free (debug_info);
        terminate = true
        #break;
        # of 1:GST_MESSAGE_EOS:
      elif gst.MessageFlag.eos in mt:
        g_print("End-Of-Stream reached.\n")
        terminate = true
        #break;
        #of 2:GST_MESSAGE_STATE_CHANGED:
      elif gst.MessageFlag.stateChanged in mt:
        # /* We are only interested in state-changed messages from the pipeline */
        # #define GST_MESSAGE_SRC(message)        (GST_MESSAGE_CAST(message)->src)
        if false: ### (GST_MESSAGE_SRC (msg) == GST_OBJECT (data.pipeline)):
          var old_state, new_state, pending_state: gst.State
          gst.parse_state_changed(msg, old_state, new_state, pending_state)
          g_print("Pipeline state changed from $1 to $2:\n" % [state_get_name(old_state), state_get_name(new_state)])
        #break;
      else: # default:
        # /* We should not reach here */
        g_printerr("Unexpected message received.\n")
        # break;
      #gst_message_unref (msg);
      if terminate:
        break
#  } while (!terminate);

  # /* Free resources */
  # gst_object_unref (bus);
  discard gst.set_state(data.pipeline, gst.State.null);
  # gst_object_unref (data.pipeline);
  return 0

discard main()

And it seems to work with ARC too, but when it terminates there is a segmentation fault unfortunately. You should debug that yourself, should be not that hard, have really to sleep now. And finally, creating a low level wrapper with futhark, nimperop or whatever may still be a better option for you -- or using just C or Rust.

$ nim c --gc:arc basic_tutorial_3.nim
Hint: used config file '/home/salewski/Nim/config/nim.cfg' [Conf]
Hint: used config file '/home/salewski/Nim/config/config.nims' [Conf]
.......................................................................................
CC: stdlib_digitsutils.nim
CC: stdlib_assertions.nim
CC: stdlib_formatfloat.nim
CC: stdlib_dollars.nim
CC: stdlib_io.nim
CC: stdlib_system.nim
CC: ../.nimble/pkgs/gintro-0.9.6/gintro/glib.nim
CC: stdlib_parseutils.nim
CC: stdlib_strutils.nim
CC: stdlib_times.nim
CC: ../.nimble/pkgs/gintro-0.9.6/gintro/gobject.nim
CC: ../.nimble/pkgs/gintro-0.9.6/gintro/gst.nim
CC: basic_tutorial_3.nim
Hint:  [Link]
Hint: gc: arc; opt: none (DEBUG BUILD, `-d:release` generates faster code)
80381 lines; 1.470s; 116.898MiB peakmem; proj: /home/salewski/mantelhere/basic_tutorial_3.nim; out: /home/salewski/mantelhere/basic_tutorial_3 [SuccessX]
salewski@nuc ~/mantelhere $ ./basic_tutorial_3 
Received new pad src_0 from source:
It has type video/x-raw which is not raw audio. Ignoring.
Received new pad src_1 from source:
Link succeeded (type audio/x-raw).
Debugging information: ../gstreamer-1.18.4/plugins/elements/gstqueue.c(990): gst_queue_handle_sink_event (): /GstPipeline:test-pipeline/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstVaapiDecodeBin:vaapidecodebin0/GstQueue:vaapi-queue:
streaming stopped, reason not-linked (-1)
No stack traceback available
SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault (core dumped)

@mantielero
Copy link
Author

Wow. This is awesome mate. It will take me more than 3h to understand what you did!

Thanks a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants