Panda::XS - useful features and typemaps for XS modules.
Panda::XS provides some useful features for XS modules. Also adds default configurable typemaps with most commonly used types. Panda::XS makes it possible for other modules (Perl or XS) to inherit from your XS module. To use it you must have a C++ compiler. Of course most (or all) CPAN modules have private implementation visible only via perl interface (function or method calls). But it is a much better approach to implement functionality in C/C++ classes with public API, make an XS interface, to make it usable from perl and also make you C code visible to other XS modules. This makes it possible for other users to use your C code directly from other XS modules, without perl method/function interface and therefore they can achieve much greater speeds. To make your C code visible to other XS modules when your module is installed, see Panda::Install.
Makefile.PL:
use strict; use Panda::Install 'write_makefile'; write_makefile( NAME => 'MyXS', CPLUS => 1, DEPENDS => 'Panda::XS', );
mytypemap.map:
MyClass* T_OPTR MyClass3* T_OEXT
MyXS.xs:
#include <xs/xs.h> /* replaces #include perl.h, ppport.h, XSUB.h, etc... */ ... # C++ class based object MyClass* MyClass::new () CODE: RETVAL = new MyClass(); ... // scalar-reference based object. IV value is C pointer address. void MyClass::somefunc () PPCODE: THIS->somefunc(); ... void MyClass::DESTROY () # hash and C++ class based object at one time using xs::sv_payload_attach; using xs::rv_payload; MODULE=... PACKAGE=MyClass2 SV* new (const char* CLASS) CODE: RETVAL = sv_bless( newRV_noinc((SV*)newHV()), gv_stashpv(CLASS, GV_ADD) ); rv_payload_attach(RETVAL, new MyClass2()); OUTPUT: RETVAL void somefunc (SV* OBJ) PPCODE: if (!SvROK(OBJ)) croak("...."); MyClass2* THIS = (MyClass2*) rv_payload(OBJ); assert(THIS); ... void DESTROY (SV* OBJ) PPCODE: if (!SvROK(OBJ)) croak("...."); MyClass2* THIS = (MyClass2*) rv_payload(OBJ); delete THIS; # ANY-SV and C++ class based object at one time MyClass3* MyClass3::new () CODE: RETVAL = new MyClass2(); void MyClass3::somefunc () PPCODE: THIS->somefunc(); ... void MyClass3::DESTROY ()
MyXS.pm:
package MyXS; use Panda::XS; our $VERSION = '0.1.3'; require Panda::XSLoader; Panda::XSLoader::bootstrap(); # replacement for XSLoader::load()
Mappings for integers
Array/Hash/Code/IO references.
AV* get_list () CODE: RETVAL = newAV(); // push values to RETVAL void merge (HV* h1, HV* h2) PPCODE: ...
Scalar/Array/Hash/IO reference based objects.
OHV* OHV::new () CODE: RETVAL = newHV(); int OAV::get_count () CODE: RETVAL = *av_fetch(THIS, 1, 1);
IV(SCALAR)-based object with C pointer attached.
TYPEMAP MyClass* T_OPTR # XS MyClass* MyClass::new () CODE: RETVAL = new MyClass(); void MyClass::somefunc() PPCODE: // use 'THIS' (MyClass*) void MyClass::DESTROY ()
The main problem of this method is that one cannot fully inherit from your module (no place to store another class' data). However you can still use inheritance XS->XS->... if your child's XS uses C++ class which inherits from parent's C++ class. In another words, if you still need to hold only single pointer. In this case, set 'basetype' to the name of the most parent C++ class. See HOW TO for details.
HOW TO
See T_OEXT for description of params.
T_OEXT
Extendable object with C pointer attached.
TYPEMAP MyClass* T_OEXT # XS MyClass* MyClass::new () CODE: RETVAL = new MyClass(); void MyClass::somefunc() PPCODE: // use 'THIS' (MyClass*) void MyClass::DESTROY ()
This method is much more flexible as it allows for inheritance and even XS -> XS inheritance where each one holds its own C++ object. I.e. it can hold an arbitrary number of pointers.
<T_OEXT C type>::new will create scalar-based object (reference to undef)
my $obj = MyClass->new; # $obj is RV to undef with C++ MyClass attached
Use obj2hv / obj2av to change the object's base to desired type.
Panda::XS::obj2hv($obj); # $obj is RV to HV with C++ MyClass attached
or
Panda::XS::obj2av($obj); # $obj is RV to AV with C++ MyClass attached
After that you can store data in $obj->{..} or $obj->[..] as if it was a regular perl object
Parameters
The most parent C++ class in XS hierarchy. Used as a marker(key) for storing C++ pointer in perl object's magic. Also used for auto typecasting from parent class to child class like this 'THIS = dynamic_cast<$type>(($basetype)stored_ptr)' for input typemaps and from child class to parent class in output typemaps 'stored_ptr = static_cast<$basetype>(RETVAL)'. By default $basetype = $type. If $basetype == $type, then casting is not performed. See HOW TO for details.
Enables automatic refcounting for C++ objects (incrementing on OUTPUT and decrementing on DESTROY). Your C++ class must either subclass panda::RefCounted class or define an interface for getting refcnt, incrementing and decrementing refcounter (see xs::refcnt_get, xs::refcnt_inc, xs::refcnt_dec).
xs::refcnt_get
xs::refcnt_inc
xs::refcnt_dec
Remember that you HAVE to define DESTROY function even if you don't have anything to do there. Otherwise, refcnt won't be decremented!
void MyClass::DESTROY ()
On OUTPUT, tries to cast retval to xs::XSBackref and if it succeedes, returns already existing SV (if any) for that object instead of creating new one every time. As a useful effect this preserves any custom perl classes that might subclass your base class as well as any perl data in object (if upgraded to AV/HV). See HOW TO for more detailed explanation.
xs::XSBackref
IMPORTANT: backref stores backreference from C++ object to SV object. To work properly it syncronizes refcounter between C++ object and SV. Therefore it REQUIRES your object to use panda::RefCounted to work out of the box. If another refcounting class is in use, you must call XSBackref::on_retain() / XSBackref::on_release() every time your object is incremented/decremented.
XSBackref::on_retain()
XSBackref::on_release()
Example using panda::RefCounted.
// MyClass - C++ class with panda::RefCounted // MyClassXS - objects of this class are created from perl class MyClassXS : public XSBackref, public MyClass { ... }
Example using custom refcounter.
// MyClass - C++ class with custom refcounter methods - rcnt_inc, rcnt_dec (they must be virtual to work properly) class MyClassXS : public XSBackref, public MyClass { virtual void rcnt_inc () const { XSBackref::on_retain(); MyClass::rcnt_inc(); } virtual void rcnt_dec () const { XSBackref::on_release(); MyClass::rcnt_dec(); } }
Wrapper to use when returning C++ object to perl. A typemap must be defined for that class and it must inherit from T_OWRAPPER typemap class. See HOW TO for examples and more details.
T_OWRAPPER
If enabled, INPUT typemap will use static_cast instead of dynamic_cast when downcasting objects. You may need this if your object doesn't have any virtual methods (no virtual table). In any other case, using static_cast is not recommended to avoid programmer's mistakes comming from perl (for example, wrong object passed: with dynamic_cast it will croak, with static_cast undefined behaviour might happen). Don't worry about perfomance, Panda::XS uses panda::dyn_cast instead of default operator. It is 10x-20x faster than dynamic_cast.
static_cast
dynamic_cast
Panda::XS
panda::dyn_cast
Callback (function) name to call on threaded perls when a new thread is created and your object is about to be copied. Function signature is:
If you dont use wrapper
<basetype> <callbackname> (<basetype> arg);
If you use wrapper
<wrapper_basetype> <callbackname> (<wrapper_basetype> arg);
See "THREADED PERLS SUPPORT" for details.
Prevents T_OEXT/T_OPTR from inserting object deletion code to the end of XS DESTROY function. Use with caution! See "DESTROY behaviour for T_OEXT and T_OPTR" for details.
Special C variables for OUTPUT
Define this variable as either 'const char*' or 'SV*' or 'HV*' and set it to a class name or a class stash (HV*) you want your object to be blessed to. It is done automatically for methods 'new', so that you must not define this variable in 'new' methods. However you can change it's value if you want to.
To receive maximum perfomance, follow these rules (in order they appear):
For example, in an object method, which returns another object, like 'clone':
HV* CLASS = SvSTASH(SvRV(ST(0)))
It's the most effective way for Panda::XS to bless your newly created object. However doing this:
HV* CLASS = gv_stashpv(classname_str, GV_ADD);
won't lead to any perfomance gain over setting const char* CLASS.
For example, in a class method, which creates object.
SV* CLASS = ST(0);
In some cases it runs much faster than setting const char* CLASS (when ST(0) is a shared COW string, since perl 5.21.4).
Don't do it for method 'new' because ExtUtils::ParseXS automatically sets "const char* CLASS = SvPV_nolen(ST(0));" which unfortunately is not the most effective way. Of course it happens only if you "typemap"ed your function (MyClass* MyClass::new ())
For example, in constructors that are called as functions, not as class methods, like 'uri("http://ya.ru")'
const char* CLASS = "MyFramework::URI";
This variable is automatically defined for all T_OEXT outputs and is set to NULL. If you don't change its value, then a new object (reference to undef) will be created and your RETVAL will be attached to it. If you want to attach your RETVAL to an already existing object, set 'self' variable to some SV* value. Value must be a valid RV to a blessed SV, or an SV itself (in this case an RV to specified SV is created and blessed). Useful for calling super and next methods. See HOW TO for details.
HV or AV based object with C pointer attached.
TYPEMAP MyClass* T_OEXT_HV # XS MyClass* MyClass::new () CODE: RETVAL = new MyClass(); void MyClass::somefunc() PPCODE: // use THIS (MyClass*) void MyClass::DESTROY () # in perl code my $obj = MyClass->new; # $obj is a blessed HASHREF $obj->somefunc;
the above works like
TYPEMAP MyClass* T_OEXT # XS MyClass* MyClass::new () CODE: RETVAL = new MyClass(); void MyClass::DESTROY () # in perl code my $obj = MyClass->new; # $obj is a blessed SCALARREF (ref to undef) Panda::XS::obj2hv($obj); # $obj is a blessed HASHREF
and exactly like
TYPEMAP MyClass* MY_TYPE OUTPUT MY_TYPE : T_OEXT if (!self && $var) self = (SV*)newHV(); INPUT MY_TYPE : T_OEXT # XS MyClass* MyClass::new () CODE: RETVAL = new MyClass(); # in perl code my $obj = MyClass->new; # $obj is a blessed HASHREF
These typemap classes always adds object deletion code to DESTROY function, even if your XS DESTROY function is non empty.
The code is:
delete THIS_wrapper;
xs::refcnt_dec(THIS); // which is THIS->release() for panda::RefCounted
// Code to release shared pointer
delete THIS;
If you don't want T_OEXT / T_OPTR to insert that for you, set prevent_default_destroy param for typemap class.
prevent_default_destroy
Keep in mind that in this case you must by any means avoid memory leaks, i.e. do the same or similar.
Here and below by 'XS CLASS' i mean not just XS code with functions, i mean class, objects of which hold a C struct or class.
Either classic pointer-based object
MyClass* T_OPTR
or more flexible "attached pointer" object
MyClass* T_OEXT
MyClass* MyClass::new () CODE: RETVAL = new MyClass(); OUTPUT: RETVAL void MyClass::somefunc () PPCODE: THIS->somefunc(); // THIS is a variable of type MyClass* ... void MyClass::DESTROY ()
my $obj = new MyClass(); $obj->somefunc();
Note how your object looks like inside with T_OPTR
DB<2> x $obj 0 MyClass=SCALAR(0x8179dc4e0) -> 34732484608
or with T_OEXT
DB<5> x $obj 0 MyClass=SCALAR(0x8179e16a8) -> undef
'Extendable' means that we can create child class in perl and hold additional data in $self as it was a regular perl object. It's impossible to implement that using T_OPTR as it makes perl object not applicable for holding anything but C pointer. To do this, we must use T_OEXT typemap class. By default, T_OEXT creates reference to undef as an object base. Most of the time we want it to be a hash reference. There are 2 ways we can achieve that - either use T_OEXT_HV as typemap class (it creates RV to HV as an object base instead of RV to undef) or upgrade already created object to RV/HV from perl.
In typemap change T_OEXT to T_OEXT_HV:
MyClass* T_OEXT_HV
That's all! Lets see the results
DB<6> x new MyClass() 0 MyClass=HASH(0x81794bf48) empty hash
Lets do the upgrade in child class.
package MyClassChild; use parent 'MyClass'; sub new { my $self = shift->SUPER::new(@_); Panda::XS::obj2hv($self); return $self; }
Lets see the results
DB<5> x new MyClass() 0 MyClass=SCALAR(0x81794bd80) -> undef DB<6> x new MyClassChild() 0 MyClassChild=HASH(0x81794a318) empty hash
Image that you need to port a C framework to perl and you want to leave it's classes structure transparent. For example, there is a C framework Foo that we need to port to perl. It has 2 classes ClassA and ClassB.
class ClassA { private: int _propA; public: ClassA (int propA = 0) : _propA(propA) {} int propA () { return _propA; } void propA (int val) { _propA = val; } virtual ClassA* clone () { return new ClassA(_propA); } } class ClassB : public ClassA { private: int _propB; public: ClassB (int propA = 0, int propB = 0) : ClassA(propA), _propB(propB) {} int propB () { return _propB; } void propB (int val) { _propB = val; } virtual ClassA* clone () { return new ClassB(_propA, _propB); } }
We want to create 2 perl classes - Foo::A which holds C ClassA class and Foo::B which holds C ClassB class. Of course Foo::B should inherit from Foo::A. What do we have to do? Almost nothing!
At first, create a custom typemap class for ClassA/ClassB hierarchy, inherit it from T_OEXT and set parameter basetype to the most parent class which will be visible in perl. In our case it is ClassA (visible via Foo::A).
basetype
ClassA* T_FOO ClassB* T_FOO OUTPUT T_FOO : T_OEXT(basetype=ClassA*) INPUT T_FOO : T_OEXT(basetype=ClassA*)
Remember that basetype is required in case of C++ inheritance, otherwise you won't be able to call Foo::A methods on a Foo::B object.
Then lets create an XS code.
Foo.xs (Here and after we use extended Panda::Install XS syntax to make XS functions look like C functions)
MODULE = Foo PACKAGE = Foo::A PROTOTYPES: DISABLE ClassA* ClassA::new (int propA) { RETVAL = new ClassA(propA); } int ClassA::propA (SV* newval = NULL) { // 'THIS' is a ClassA* pointer, even if real perl object is of Foo::B class. if (newval) THIS->propA(SvIV(newval)); RETVAL = THIS->propA(); } ClassA* ClassA::clone () { HV* CLASS = SvSTASH(SvRV(ST(0))); RETVAL = THIS->clone(); } void ClassA::DESTROY () {} MODULE = Foo PACKAGE = Foo::B PROTOTYPES: DISABLE ClassB* ClassB::new (int propA, int propB) { RETVAL = new ClassB(propA, propB); } int ClassB::propB (SV* newval = NULL) { // 'THIS' is a ClassB* pointer if (newval) THIS->propB(SvIV(newval)); RETVAL = THIS->propB(); }
Somewhere in Perl
my $a = new Foo::A(123); say $a->propA; $a->propA(321); say ref $a->clone; # Foo::A my $b = new Foo::B(1,2); say $b->propA; say $b->propB; say ref $b->clone; # Foo::B
Using these technics you can port any C++ framework saving its original OOP structure, keeping your XS wrapper as thin as possible and therefore get maximum perfomance.
It works out of the box. Lets add a third class to previous example, ClassC:
class ClassC { public: ClassC () {} int calculate (ClassA* arg) { return arg->propA * arg->propA; } }
Add it to a typemap
ClassC* T_OEXT
Add XS for ClassC
MODULE = Foo PACKAGE = Foo::C PROTOTYPES: DISABLE ClassC* ClassC::new () int ClassC::calculate (ClassA* arg) void ClassC::DESTROY () {}
Just do it in perl as you would in C++, typemaps will do all the work for you
my $a = new Foo::A(10); my $b = new Foo::B(20, 30); my $c = new Foo::C; say $c->calculate($a); # 100; say $c->calculate($b); # 400;
Remember, everywhere you pass/receive Foo::A, Foo::B etc, in XS you receive/return ClassA*, ClassB*, etc
Sometimes source code of parent class is unavailable, and therefore you can't make a C++ child class as described above. However you can still provide such inheritance on perl level. All you need to do is to call SUPER in constructor.
Suppose we have two C++ classes Bow and Milk. And we already have an XS class for Bow. We want to implement XS for Milk and inherit it from Bow. As we want 2 different C data pointers in one perl object, we should either not provide a basetype parameter in typemap, or set it to something different from Bow*. By default (if not provided), basetype is set to the value of final type name. However we need to call SUPER::new, and attach our data to existing SV, rather than creating new one (otherwise Bow's data would be lost). To do this, set 'self' variable to SV object to attach to. Unfortunately, as of current perl (5.21.5), it's impossible to call SUPER/next from XS. To call SUPER/next, use the API provided by Panda::XS C++ interface - call_super.
Typemap:
Bow* T_OEXT Milk* T_OEXT
Milk.pm:
package Foo::Milk; use parent 'Some::Bow';
XS:
MODULE = Foo PACKAGE = Foo::Milk PROTOTYPES: DISABLE Milk* Milk::new (...) { self = xs::call_super(cv, &ST(0), items); RETVAL = new Milk(); } void Milk::something () { // 'THIS' is a Milk* pointer } Milk* Milk::clone () { // cloning just Milk* without Bow* does NOT make sense. So we must call SUPER::clone HV* CLASS = SvSTASH(SvRV(ST(0))); self = xs::call_super(cv, &ST(0), items); RETVAL = THIS->clone(); } // new Foo::Milk object returned containing cloned Milk* and Bow* classes. void Milk::DESTROY () { xs::call_super(cv, &ST(0), items, G_DISCARD); }
All you need to do is to call next::method or maybe::next::method from your construtor. To do this, use call_next_method() or call_next_maybe() from xs:: namespace.
MyPlugin* T_OEXT
Perl:
package Foo::MyPlugin; use mro 'c3';
MODULE = Foo PACKAGE = Foo::MyPlugin PROTOTYPES: DISABLE MyPlugin* MyPlugin::new (...) { self = xs::call_next_maybe(cv, &ST(0), items); RETVAL = new MyPlugin(); } uint32_t MyPlugin::something () { // 'THIS' is a MyPlugin* pointer RETVAL = THIS->something; } MyPlugin* MyPlugin::clone () { HV* CLASS = SvSTASH(SvRV(ST(0))); self = xs::call_next_maybe(cv, &ST(0), items); RETVAL = THIS->clone(); } // new Foo::MyPlugin object returned containing cloned MyPlugin* and all data of other classes void MyPlugin::DESTROY () { xs::call_next_maybe(cv, &ST(0), items, G_DISCARD); }
package MyServer; use parent qw/Foo::MyPlugin Foo::Milk/; my $obj = new MyServer();
Sometimes you need to hold additional info in XS class, but you can't or don't want to create additional fields in your C++ class. Let's see an example. Assume we have a 'Driver' C++ class that we want to port. We'll make Foo::Driver perl class.
#TYPEMAP Driver* T_OEXT #XS Driver* Driver::new (int arg1, char* arg2) { RETVAL = new Driver(arg1, arg2); } int Driver::category () { RETVAL = THIS->category; } char* Driver::name () { RETVAL = THIS->name; } void Driver::ban (int why) { THIS->ban(why); } HV* Driver::get_history () { DriverHistory* history = THIS->get_history(); RETVAL = newHV(); // fill RETVAL with data from DriverHistory* } void Driver::DESTROY () {}
Now imagine that we want to count, how many times a driver has been banned for particular object. But Driver class doesn't have any field to hold this value. Adding it to Driver, even if you can, would be a bad decision as it shouldn't know anything about your XS class behaviour. Also filling up HV every time 'get_history' is called, could be quite expensive, so that we also would like to cache once built result somewhere. How to do that?
class DriverXS : public Driver { public: int counter; HV* cached_history; DriverXS (int arg1, char* arg2) : counter(0), cached_history(NULL), Driver(arg1, arg2) {} ~DriverXS () { SvREFCNT_dec(cached_history); } }
Now use your child class instead of Driver
#TYPEMAP DriverXS* T_OEXT #XS DriverXS* DriverXS::new (int arg1, char* arg2) { RETVAL = new DriverXS(arg1, arg2); } int DriverXS::category () { RETVAL = THIS->category; } char* DriverXS::name () { RETVAL = THIS->name; } void DriverXS::ban (int why) { THIS->counter++; THIS->ban(why); } HV* DriverXS::get_history () { if (!THIS->cached_history) { DriverHistory* history = THIS->get_history(); THIS->cached_history = newHV(); // fill cached_history with data from DriverHistory* } RETVAL = THIS->cached_history; } void DriverXS::DESTROY () {}
Although this will work out for our case, this approach may be unacceptable. For example imagine we have DriverStorage in C++ lib and class method get_driver_by_id. How can we port it? Lets create XS class Foo::DriverStorage. And method get_driver_by_id:
DriverXS* DriverStorage::get_driver_by_id (int id) { RETVAL = THIS->get_driver_by_id(id); }
But wait a minute, our c++ lib's get_driver_by_id returns Driver* of course, as it doesn't know anything about DriverXS. And even if we stored only DriverXS* objects in DriverStorage, some C++ code might have created a simple Driver* object and store it in DriverStorage. So the example above won't even compile. The only way is to create a copy DriverXS* from Driver*.
DriverXS* DriverStorage::get_driver_by_id (int id) { Driver* driver = THIS->get_driver_by_id(id); if (!driver) XSRETURN_UNDEF; RETVAL = new DriverXS(driver->category, driver->name); // or new DriverXS(driver) if have such constructor }
But it might be expensive and not every object can be copied (for example, socket handle classes).
class XSDriverWrapper { public: Driver* obj; int counter; HV* cached_history; XSDriverWrapper (Driver* driver) : counter(0), cached_history(NULL), obj(driver) {} ~XSDriverWrapper () { SvREFCNT_dec(cached_history); delete obj; } } #TYPEMAP XSDriverWrapper* T_OEXT #XS XSDriverWrapper* XSDriverWrapper::new (int arg1, char* arg2) { RETVAL = new XSDriverWrapper(new Driver(arg1, arg2)); } int XSDriverWrapper::category () { RETVAL = THIS->obj->category; } char* XSDriverWrapper::name () { RETVAL = THIS->obj->name; } void XSDriverWrapper::ban (int why) { THIS->counter++; THIS->obj->ban(why); } HV* XSDriverWrapper::get_history () { if (!THIS->cached_history) { DriverHistory* history = THIS->obj->get_history(); THIS->cached_history = newHV(); // fill cached_history with data from DriverHistory* } RETVAL = THIS->cached_history; } void XSDriverWrapper::DESTROY () {} ... XSDriverWrapper* DriverStorage::get_driver_by_id (int id) { Driver* driver = THIS->get_driver_by_id(id); if (!driver) XSRETURN_UNDEF; RETVAL = new XSDriverWrapper(driver); }
This approach is much more flexible as it allows you to return perl objects even for c++ objects who wasn't created in XS class. But our code is not good enough for now, because it's very annoying to write THIS->obj. Moreover, if you share your module with someone, they will have to know about your wrapper to work with your XS objects. For example, image you've uploaded Foo to CPAN. Someone is using it:
write_makefile( ... BIN_DEPS => ['Foo'], # make C headers, and typemaps of Foo visible to my XS/C code ); # some user's XS void change_drivers_name (XSDriverWrapper* driver_wrapper, const char* newname) PPCODE: Driver* driver = driver_wrapper->obj; driver->set_name(newname); # perl code my $driver = Foo::Driver->new(1, "vasya"); Bar::change_drivers_name($driver, "petya"); say $driver->name;
No one should know the details of your XS module (about the fact that you wrap real Driver into XSDriverWrapper). To do that, we need to implement all the details in typemap, not in XS functions itself. We will need to create our own typemap class.
#TYPEMAP XSDriverWrapper* XT_FOO_DRIVER_WRAPPER Driver* XT_FOO_DRIVER OUTPUT XT_FOO_DRIVER_WRAPPER : T_OWRAPPER $arg = new XSDriverWrapper($var); XT_FOO_DRIVER : T_OEXT(wrapper=XSDriverWrapper*) INPUT XT_FOO_DRIVER_WRAPPER : T_OWRAPPER $var = $arg->obj; XT_FOO_DRIVER : T_OEXT(wrapper=XSDriverWrapper*)
Here we use wrapper feature of T_OEXT/T_OPTR typemap classes. It allows us to wrap an object into another and make it fully transparent to XS code. Variables in XS code ('THIS' for example) will point to the object itself, not the wrapper. If you need access to the wrapper, use _wrapper extension, for example THIS_wrapper.
wrapper
_wrapper
THIS_wrapper
Now our code looks ok:
Driver* Driver::new (int arg1, char* arg2) { RETVAL = new Driver(arg1, arg2); } int Driver::category () { RETVAL = THIS->category; } char* Driver::name () { RETVAL = THIS->name; } void Driver::ban (int why) { THIS_wrapper->counter++; THIS->ban(why); } HV* Driver::get_history () { if (!THIS_wrapper->cached_history) { DriverHistory* history = THIS->get_history(); THIS_wrapper->cached_history = newHV(); // fill cached_history with data from DriverHistory* } RETVAL = THIS_wrapper->cached_history; } void Driver::DESTROY () {} ... Driver* DriverStorage::get_driver_by_id (int id) { RETVAL = THIS->get_driver_by_id(id); }
Now the implementation details are hidden from users of your module. And the third-party example above will now look much clearer:
void change_drivers_name (Driver* driver, const char* newname) PPCODE: driver->set_name(newname);
Users of your module no longer see XSDriverWrapper.
NOTE: when using wrapper feature of T_OEXT/T_OPTR, your wrapper's typemap class must inherit from T_OWRAPPER. Don't use wrapper's typemap in XS code, it won't work!
void XSDriverWrapper::myfunc (XSDriverWrapper* other) { // WRONG!!! WON'T COMPILE! THIS->counter = other->counter; }
You should always use the object itself and access wrapper via variable ${var}_wrapper.
${var}_wrapper
void Driver::myfunc (Driver* other) { // CORRECT THIS_wrapper->counter = other_wrapper->counter; }
While you only have single XS classes without any relationships with each other, everything's ok. Problems begin in the following example (we use simple (without wrapper) Driver* class and XS from previous examples):
class Insurance { private: Driver* _first_driver; Driver* _second_driver; public: Insurance () { _first_driver = new Driver(1, "john"); _second_driver = new Driver(2, "mike"); } Driver* first_driver () { return _first_driver; } Driver* second_driver () { return _second_driver; } void first_driver (Driver* val) { _first_driver = val; } void second_driver (Driver* val) { _second_driver = val; } ~Insurance () { delete _first_driver; delete _second_driver; } } #XS Insurance* Insurance::new () { RETVAL = new Insurance(); } Driver* Insurance::first_driver (Driver* newval = NULL) : ALIAS(second_driver=1) { if (newval) { if (ix == 0) THIS->first_driver(newval); else THIS->second_driver(newval); XSRETURN_UNDEF; } const char* CLASS = "Foo::Driver"; if (ix == 0) RETVAL = THIS->first_driver(); else RETVAL = THIS->second_driver(); } void Insurance::DESTROY () {}
So where is the problem? Right here:
#perl code my $insurance = new Foo::Insurance; say $insurance->first_driver->name; # prints "john" say $insurance->first_driver->name; # OOPS. core dump or undefined behaviour
When first_driver() is called for the first time, it creates perl object and attaches Driver* object to it. After first say(), the temporary perl variable holding Driver perl object no longer needed and destroyed. Destructor (DESTROY) of Driver's XS class destroys Driver* object as well. Pointer to that freed object still remains in Insurance* object and bad things happen. Ok, lets stop deleting Driver* object from Driver::DESTROY(), i.e. remove this function:
void Driver::DESTROY () {}
In this case we get a memory leak for this type of code:
for (1..1000) { my $driver = new Foo::Driver(1, "myname"); }
How can we avoid deletion in first case and memory leak in second?
We need to add a reference counter to our objects!
class Driver { private: mutable int _refcnt; .... Driver (...) : _refcnt(0) ... { ... } void retain () const { _refcnt++; } void release () const { if (--_refcnt <= 0) delete this; } virtual ~Driver () {...} } class Insurance { ... void first_driver (Driver* val) { if (_first_driver) _first_driver->release(); _first_driver = val; if (_first_driver) _first_driver->retain(); } } #XS Driver* Driver::new (...) { RETVAL = new Driver(...); RETVAL->retain(); ... } void Driver::DESTROY () { THIS->release(); } Driver* Insurance::first_driver (Driver* newval = NULL) : ALIAS(second_driver=1) { ... if (ix == 0) RETVAL = THIS->first_driver(); else RETVAL = THIS->second_driver(); if (RETVAL) RETVAL->retain(); }
We can implement it in typemap to hide these details from XS code.
#TYPEMAP Driver* XT_FOO_DRIVER OUTPUT XT_FOO_DRIVER : T_OEXT $var->retain(); INPUT XT_FOO_DRIVER : T_OEXT DESTROY { $var->release(); } #XS Driver* Driver::new (...) { RETVAL = new Driver(...); ... } void Driver::DESTROY () {} Driver* Insurance::first_driver (Driver* newval = NULL) : ALIAS(second_driver=1) { ... if (ix == 0) RETVAL = THIS->first_driver(); else RETVAL = THIS->second_driver(); }
Note, however, that you need to retain() and release() objects in C++ classes code itself properly as in any other refcounted system.
T_OEXT/T_OPTR can do all of these automatically if you set refcnt param. In this case your class must either subclass panda::RefCounted class which provides refcnt/retain/release functionality or add an API for typemap.
refcnt
Here is an example of our case with refcnt feature.
#TYPEMAP Driver* XT_FOO_DRIVER OUTPUT XT_FOO_DRIVER : T_OEXT(refcnt) INPUT XT_FOO_DRIVER : T_OEXT(refcnt) # XS and C++ class remains the same # somewhere in our code we must define an API inline int32_t xs::refcnt_get (const Driver* var) { return var->refcnt(); } inline void xs::refcnt_inc (const Driver* var) { var->retain(); } inline void xs::refcnt_dec (const Driver* var) { var->release(); }
Instead of adding refcnt functionally to your Driver class as we did, you can just inherit from panda::RefCounted class. In this case you should not define any API functions.
panda::RefCounted
class Driver: virtual public RefCounted { ... } # all code is the same
Remember: if you use wrapper, it should NOT have a refcounter. Make the underlying object itself refcounted. Wrappers are stricly bound to their SV object and should live in sync with it.
The second solution of the problem above is shared pointers. It allows you to solve this problem even if you can't change sources of C++ classes.
You can use either STL's std::shared_ptr or panda::shared_ptr (STL compatible shared pointer with intrusive mode support).
T_OEXT/T_OPTR automatically detects if you use shared pointers. Everything will work as you expect.
typedef shared_ptr<Driver> DriverSP; #typemap DriverSP T_OEXT #code class Driver { // we don't need to add anything private: Driver (...) : ... { ... } virtual ~Driver () {...} } class Insurance { private: DriverSP _first_driver; public: DriverSP first_driver () { return _first_driver; } void first_driver (DriverSP driver) { _first_driver = driver; } } #XS DriverSP new (SV* CLASS, ...) { RETVAL = DriverSP(new Driver(...)); ... } void DESTROY (DriverSP THIS) {} ... DriverSP Insurance::first_driver (DriverSP newval = DriverSP()) : ALIAS(second_driver=1) { ... if (ix == 0) RETVAL = THIS->first_driver(); else RETVAL = THIS->second_driver(); }
You might have noted several things:
Typemap class is bound to shared pointer's type, not to an object's type. Therefore you must set a shared pointer to OUTPUT variable (RETVAL), and you will receive a shared pointer in INPUT args
Shared pointers are passed by value, not by pointer. See how we receive an INPUT arg 'THIS':
const char* name (DriverSP THIS) // CORRECT const char* name (DriverSP* THIS) // WRONG!!! const char* DriverSP::name () // WRONG!!!
The third example won't work because for ExtUtils::ParseXS it's just an alias for
const char* name (DriverSP* THIS)
:((
Using shared pointers introduces slight overheat, as it have to alloc additional memory per object. However, see next paragraph to find out how you can remove this overheat while still using shared pointers.
Remember: if you use wrapper, don't use it via shared pointers, make shared pointers for objects itself and hold it in wrapper.
If you decided to use panda::shared_ptr instead of std::shared_ptr, you can get some benefit if you inherit your class from panda::RefCounted or by any other means implement refcounting via retain/release methods. panda::shared_ptr automatically detects classes with retain/release and works differently.
Firstly, it doesn't malloc anything, as your class already has a refcounter. It just calls retain() and release().
retain()
release()
Secondly, as you know, creating second std::shared_ptr for an object is a critical mistake. However, for our case it isn't true. You can freely create second, third, etc shared_ptr for any object with retain/release methods without losing reference counter.
Thirdly, sizeof(panda::shared_ptr<ClassWhichHasRetainRelease)> is just one pointer.
sizeof(panda::shared_ptr<ClassWhichHasRetainRelease
That means you can freely interchange MyClass* with MyClassSP and vice versa.
For example:
typedef shared_ptr<Driver> DriverSP; #typemap Driver* T_OEXT(refcnt) DriverSP T_OEXT #code class Driver : virtual public panda::RefCounted { private: Driver (...) : ... { ... } virtual ~Driver () {...} } class Insurance { private: DriverSP _first_driver; public: DriverSP first_driver () { return _first_driver; } void first_driver (DriverSP driver) { _first_driver = driver; } } #XS Driver* Driver::new (...) { RETVAL = new Driver(...); ... } void Driver::DESTROY () {} ... # See how we receive DriverSP, while Driver object is T_OEXT(refcnt) (i.e. Driver*) # It would be a fatal logical error doing this for shared pointers which wraps non-refcounted objects. DriverSP Insurance::first_driver (DriverSP newval = DriverSP()) : ALIAS(second_driver=1) { ... if (ix == 0) RETVAL = THIS->first_driver(); else RETVAL = THIS->second_driver(); }
Moreover you don't need DriverSP T_OEXT typemap at all, as it would automatically convert it to and from shared pointer.
DriverSP T_OEXT
And mainly remember, do all of the above in this paragraph ONLY if your class implements refcounting. Otherwise you would split the reference counter into many instances and bad things would surely happen.
Consider the following example (using our lovely Driver and Insurance classes with refcnt):
my $insurance = new Foo::Insurance; my $driver = new Foo::Driver(1, 'smith'); $insurance->first_driver($driver); my $same_driver = $insurance->first_driver; my $same_driver2 = $insurance->first_driver; say "SAME" if $driver eq $same_driver; say "SAME" if $same_driver eq $same_driver2;
Nothing is printed in our example, because every time we return Driver* to perl, typemap creates a new perl object and associates it with C++ object. Of course those 2 objects hold the same C++ object, so most of the time it's not a problem at all.
However if you plan that users of your module will extend your classes, the problem appears:
package MyDriver; use parent 'Foo::Driver'; sub new { my $self = shift->SUPER::new(@_); Panda::XS::obj2hv($self); $self->{id} = 123; } sub id { $_[0]->{id} } package main; my $insurance = new Foo::Insurance; my $driver = new MyDriver(1, 'smith'); $insurance->first_driver($driver); my $same_driver = $insurance->first_driver; say ref $same_driver; # prints 'Foo::Driver';
Perl object returned is of a default class (without custom data created in MyDriver::new()), because our XS function Driver* Insurance::first_driver says const char* CLASS = "Foo::Driver". And that's generally ok, because it doesn't know anything about your MyDriver perl class which was the owner of the C++ Driver* object, it only holds C++ Driver* objects which are the same for both MyDriver and the default Foo::Driver.
Driver* Insurance::first_driver
const char* CLASS = "Foo::Driver"
MyDriver
Foo::Driver
To link C++ object to a particular SV object we must use backref technique. To do this, make your C++ object extend xs::XSBackref class and set backref parameter for typemap class.
backref
class XSDriver : public Driver, public xs::XSBackref { public: XSDriver (int arg1, char* arg2) : Driver(arg1, arg2) {} } # TYPEMAP Driver* XT_FOO_DRIVER OUTPUT XT_FOO_DRIVER : T_OEXT(basetype=Driver*, refcnt, backref) INPUT XT_FOO_DRIVER : T_OEXT(basetype=Driver*, refcnt, backref) # XS Driver* Driver::new (int arg1, char* arg2) { RETVAL = new XSDriver(arg1, arg2); }
That's all! Check it out now:
my $insurance = new Foo::Insurance; my $driver = new MyDriver(1, 'smith'); $insurance->first_driver($driver); my $same_driver = $insurance->first_driver; say ref $same_driver; # prints 'MyDriver'; say $same_driver->id; # prints 10
You can even loose all references to perl object from perl:
my $insurance = new Foo::Insurance; my $driver = new MyDriver(1, 'smith'); $insurance->first_driver($driver); undef $driver; my $same_driver = $insurance->first_driver; say ref $same_driver; # prints 'MyDriver'; say $same_driver->id; # prints 10
As you can see it won't die until there are any references from perl or C++. That's because XSBackref syncronizes perl's refcounter and C++'s refcounter.
As a side effect, if using backref, performance of such methods increases, because perl object is no longer created every time you return C++ object to perl.
Note how we created XSDriver to add XSBackref to parent classes. We had an option to change Driver class itself and add XSBackref to its parents, however it wouldn't be a good decision, because Driver is a pure C class and should not know anything about perl.
XSDriver
XSBackref
Driver
Keep in mind that not every C++ object in our example extends XSBackref. For example, those 2 who are created in Insurance's constructor are just Driver*. Typemap automatically detects objects which don't extend XSBackref and reverts to default behaviour (without backref param). In this case const char* CLASS in XS method is in effect, and perl objects of that class will be returned every time you call method (like it would without backref param). For example:
Driver*
const char* CLASS
my $insurance = new Foo::Insurance; # fills first and second driver in constructor with base Driver* objects my $driver = $insurance->first_driver; my $same_driver = $insurance->first_driver; say ref $driver; # Foo::Driver say $driver eq $same_driver; # false $driver->name('abcd'); say $driver->name eq $same_driver->name; # true, as they are still holding the same C++ object
In order to use backref your class MUST use refcounting, because backreferencing is impossible without reference counting. If you use panda::RefCounted, you have nothing to do additionally. If you use other refcounting class, you have to call XSBackref::on_retain() and XSBackref::on_release() every time your object is retained or released. If you use panda::RefCounted, you should use virtual inheritance.
If you use wrapper, the object itself must extend XSBackref, NOT the wrapper because it doesn't make sense.
If you support threaded perls, you must choose either CLONE_SKIP strategy or clone strategy. Sharing C++ object between threads is impossible because backref cannot point to many different SV objects (each for particular interpreter). See "THREADED PERLS SUPPORT" for details.
CLONE_SKIP
clone
Mix-class for XS backreferences. See HOW TO. You will also have to enable backref param in typemap.
Public property where original perl object is stored. It is not an RV to object, it is a blessed object itself.
If you have custom refcounter, call this method when your object is retained.
If you have custom refcounter, call this method when your object is released.
Functions marked with [pTHX] must receive aTHX as a first arg.
[pTHX]
aTHX
Registers perl package module in %INC as if it was defined in source_module.
module
%INC
source_module
Image you have XS module My::Module and it has My::Module::Class inside and it is fully defined in XS code and has no Class.pm file. In such a case user can't say
use My::Module; use My::Module::Class; # croaks because it's not loaded and no My/Module/Class.pm on disk.
One can answer why does user need to say use My::Module::Class ? After all all packages has already been imported because use My::Module loaded the XS module itself. However user sometimes need to import constants or functions:
use My::Module::Class
use My::Module
use My::Module::Class qw/CONSTANT function/;
This functions registers module so that the call above no longer tries to load perl .pm file from disk, only calls import().
import()
If source_module does not exist, function does nothing and false is returned. Otherwise function returns true.
false
true
Makes module inherit from parent. Adds parent to the end of @module::ISA. The reason why one may need to call it from XS is the same as explained in register_package function: when you don't have perl .pm wrapper around some XS package.
parent
@module::ISA
register_package
Creates panda::string from SV string.
Panda::XS installs a typemap for panda::string, so it is okay to receive it in XS function.
using panda::string; ... void myfunc (string str) PPCODE: printf("string is %s, len is %d", str.data(), str.length()); ...
Creates string_view from SV string.
Panda::XS installs a typemap for string_view, std::string_view so it is okay to receive it in XS function.
Attach payload ptr to sv. Markers allow you to store multiple payloads in a single SV. Marker doesn't need to be initialized somehow, its just a valid memory pointer. Usually its a static global variable. If no marker provided, default one is used (global to the whole program).
Returns true if sv has any payload for marker.
Returns payload attached to sv.
Removes payload from sv and returns number of payloads removed.
Same as sv_* but operates on sv referenced by a given rv. Doesn't check if a given RV is valid.
Static marker for particular class. Used internally by T_OEXT typemap class.
Same as above but returns marker which attaches svdup_callback to magic.
svdup_callback
svdup_callback is a callback to run on threaded perls when a thread is created. Called for each object and receives magic struct attach to an object with current marker. Return value is ignored. Used to clone or do something other with payload attached to this magic. Note that you don't need to clone SV* payload, as threaded perl duplicates it by itself.
typedef int (*on_svdup_t) (pTHX_ MAGIC* mg, CLONE_PARAMS* param);
Returns a marker in runtime for specified key. If no marker exists for that key, creates and returns it. Performs hash searching, so that in most cases you should put the result into a static variable to prevent calling this function many times.
Note that marker SVPayloadMarker<MyClass*::marker> is NOT the same marker as sv_payload_marker("MyClass*").
SVPayloadMarker<MyClass*
sv_payload_marker("MyClass*")
Calls super method from inside XS method. 'cv' is a pointer to currently running XS function (every XS function has this variable). 'args' and 'items' describes perl variables to pass to super method. If you want to passthrough all the arguments your XS function received, pass '&ST(0)' and 'items'. 'flags' is passed to Perl's API call_method / call_sv as is. SUPER is always called in SCALAR context. Returned SV* has its refcounter increased. If you don't need it, decrement it. If you don't need return value at all, pass G_DISCARD in flags.
Calls next method (C3 MRO). See "call_super" for argument details.
Calls next method (C3 MRO). Return undef if no next method. See "call_super" for argument details.
Generic form of call_super/call_next_method/call_next_maybe. 'type' is either xs::NEXT_SUPER or xs::NEXT_METHOD or xs::NEXT_MAYBE.
Calls perl sub cv in void context with arguments args where <items> is the number of elements in <args>. Everything sub actually returns is discarded.
cv
args
CV* cv = get_cv("Module::func", 0); if (!cv) croak("no sub"); SV* args[] = {arg1, arg2}; xs::call_sub_void(cv, args, 2);
Calls perl sub cv in scalar context with arguments args. Returns the resulting scalar. If subroutine returns more than one value, the rest is discarded. Return value is mortal, so you must either finish using it before next FREETMPS occur (next perl func call, XS function return, and so on), or SvREFCNT_inc it. flags are passed to call_sv. If G_DISCARD is set, then discards the return value and returns NULL (the difference with call_sub_void is that the subroutine is called in scalar context, not void).
SvREFCNT_inc
flags
call_sv
call_sub_void
CV* cv = get_cv("Module::func", 0); SV* args[] = {arg1, arg2}; SV* ret = xs::call_sub_scalar(cv, args, 2); if (ret && need_save_ret) myobj->val = SvREFCNT_inc(ret);
Calls perl sub cv in list context with arguments args and places the resulting values in ret. maxret is the maximum number of values you wish to receive from this call. Returns the number of resulting values actually written to ret. Every <SV*> in array is mortal, so you must either finish using it before next FREETMPS occur (next perl func call, XS function return, and so on), or SvREFCNT_inc it. If subroutine returns more than maxret values then the rest is discarded and returns maxret. Therefore you can't receive arbitrary large number of return values with this function. flags are passed to call_sv. If G_DISCARD is set, then discards all of the returned values and returns 0 (the difference with call_sub_void is that the subroutine is called in list context, not void). ret can be NULL if either maxret is 0 or G_DISCARD is set.
ret
maxret
CV* cv = get_cv("Module::func", 0); SV* args[] = {arg1, arg2}; SV* ret[3]; I32 nret = xs::call_sub_list(cv, ret, 3, args, 2); for (int i = 0; i < nret; ++i) sv_dump(ret[i]);
Does the same as previous function, however pushes all of returned values in perl array and returns it. If subroutine returned zero values, returns NULL. THe returned array is mortal, so you must either finish using it before next FREETMPS occur (next perl func call, XS function return, and so on), or SvREFCNT_inc it. flags are passed to call_sv. If G_DISCARD is set, discards everything returning NULL.
CV* cv = get_cv("Module::func", 0); AV* ret = xs::call_sub_list(cv); if (ret) XS_AV_ITER(ret, { sv_dump(elem); });
Calls method name with length len on object obj in void context. If object is not a blessed object, calls class method name assuming that obj is a class name. Everything else behaves like call_sub_void.
name
len
obj
xs::call_method_void(obj, "init", 4);
Calls method name with length len on object obj in scalar context. If object is not a blessed object, calls class method name assuming that obj is a class name. Everything else behaves like call_sub_scalar.
call_sub_scalar
SV* ret = xs::call_method_scalar(obj, "name", 4);
Calls method name with length len on object obj in list context. If object is not a blessed object, calls class method name assuming that obj is a class name. Everything else behaves like call_sub_list.
call_sub_list
SV* ret[10]; I32 nret = xs::call_method_list(obj, "get_phones", 10, ret, 10); for (int i = 0; i < nret; ++i) sv_dump(ret[i]);
AV* list = xs::call_method_av(obj, "get_phones", 10); if (list) XS_AV_ITER(list, { sv_dump(elem); });
Iterates through perl hash very fast. On each iteration executes code, current hash entry is accessible via HE* he variable. Use perl API's HeVAL, HeKEY, HeKLEN to extract value and key from hash entry.
code
HE* he
HeVAL
HeKEY
HeKLEN
hv must be a non-null HV* variable. Doesn't handle tied hashes.
hv
Same as XS_HV_ITER but skips entries whose values are undefs. _NU stands for 'not undef'.
XS_HV_ITER
Iterates through perl array very fast. On each iteration executes code, current array element is accessible via SV* elem variable. Keep in mind that it might be a NULL pointer in case if current slot in array is empty.
SV* elem
For example, in this case
my @arr; $arr[1] = undef; $arr[3] = 10;
slots 0 and 2 are empty. Slot 1 is not empty but it's value is undefined. Don't confuse empty slot with undefined values, empty slot is an absence of SV* variable, while undefined value is a legal SV* variable for which SvOK(elem) returns false.
av must be a non-null AV* variable. Doesn't handle tied arrays.
av
Same as XS_AV_ITER but skips empty slots. _NE stands for 'not empty'.
XS_AV_ITER
Same as XS_AV_ITER but skips empty slots and undefined emelents. _NU stands for 'not undef'.
Macro that catches c++ exceptions and croaks (throws perl exceptions). Example
#include <xs/xs.h> // somewhere in code PXS_TRY({ throw std::logic_error("hello"); }); // in perl code MyXSModule::function_that_leads_to_the_code_above(); // possible output "[std::logic_error] hello" at misc/mytest.plx line 30.
Output depends on exception type:
[FQN_exception_class_name] exc.what()
string in const char*
value.c_str()
unknown c++ exception thrown
Returns perl string '[FQN_exception_class_name] exc.what()'
Upgrades $obj to HASHREF or ARRAYREF.
$obj must be a valid reference to something lower than array or hash (undef, number, string, etc). Otherwise will croak. If $obj is already of desired type, nothing is done.
Attach $payload to $target. $target can be any l-value scalar. If $target has any $payload, it gets removed.
Returns payload attached to $target if any, otherwise undef.
Returns true if $target has any payload.
Removes payload from $target. Returns true if any payload has been removed.
Same as sv_* but $target must be reference to perl object.
Same as sv_* or rv_*, but if $target is ref then payload will be attached/detached from referenced object, otherwise from $target itself.
If you don't do anything special, on threaded perls after thread creation you will most likely core dump. The reason is that when perl creates a thread it duplicates all perl variables available in current interpreter. So that you will likely get 2 perl variables holding a pointer to the same C/C++ object (refcnt won't help - nobody calls retain()). When these objects are destroyed, double delete might occur on the same C object. That's bad.
delete
You have 4 options to handle that:
To do this, just add CLONE_SKIP method to your class and return a true value from it. In this case perl won't clone SV variable holding your object - it will place an unblessed reference to undef instead. Of course users must know about this behaviour, because they won't be able to use objects of your class created before new thread. However they can still create new objects from new thread and everything will be ok.
This is the only option for you if you use T_OPTR. T_OEXT doesn't have this limitation.
To do this, don't define CLONE_SKIP method, instead set on_svdup parameter of T_OEXT class to 'clone'. In this case when a thread creates, method clone() will be called on your C++ object and this new object is stored in duplicated variable. Of course you must have a clone() method with no args in your class. It must be virtual if you support inheritance. For example:
on_svdup
clone()
T_MYCLASS : T_OEXT(basetype=MyClass*, on_svdup=clone)
Note that if you use wrapper feature, then clone() will be called on wrapper, not on the object. You must clone your object from wrapper's clone().
If you use backref feature, backreference in your object (which still points to the old SV from the parent thread) will be changed automatically after clone, you don't need to do something special for that (even if you use wrapper).
To do this, don't define CLONE_SKIP method, instead set on_svdup parameter of T_OEXT class to 'retain'. In this case when a thread creates, method retain() will be called on your C++ object and the same object is stored in duplicated variable. Of course you must have a retain() method with no args in your class (if you subclass panda::RefCounted, you do).
T_MYCLASS : T_OEXT(basetype=MyClass*, on_svdup=retain)
Note that if you use wrapper, you should set on_svdup=clone and do retain() of the object in wrapper's clone(). That's because wrappers are strictly bound to their SV (one-to-one) and cannot be shared, because it doesn't make sense.
on_svdup=clone
You CANNOT use backref, because we must use different SV in each thread, however if you retain you object, we would have only single pointer for backreference. So that if you need backref you should clone your objects, not retain.
retain
To do this, don't define CLONE_SKIP method, instead set on_svdup parameter of T_OEXT class to the name of your callback which will handle thread creation for objects of your class. Callback must have the following signature:
<basetype> <callbackname> (pTHX_ <basetype> arg);
Callback will be passed current C object. Callback shall return a C object to be set to the duplicated perl variable (it may be a new one or the same object). Keep in mind that if your module shares API for other modules, this callback MUST be externally visible, because when someone uses your class as a return type from XS functions, typemap will insert code, containing your callback name into those XS functions. Don't forget to specify it with full qualified name for the same reason. Example:
# in .h namespace my { MyClass* myclass_onsvdup (pTHX_ MyClass* old); } # in .cc MyClass* my::myclass_onsvdup (pTHX_ MyClass* old) { return new MyClass(old->data); } # in typemap T_MYCLASS : T_OEXT(basetype=MyClass*, on_svdup=my::myclass_onsvdup)
Note that if you use wrapper feature, the return and input types of the callback are basetype of wrapper itself, not the object.
If you use backref, then backreference is changed automatically in returned object to point new SV no matter if it is the same object or not (even with wrapper). Keep in mind that if you implement some kind of retain strategy in your callback, you will break backref. Bad thing will surely happen.
Perl has 2 options for XS when working on threaded perls:
1) Not defining PERL_NO_GET_CONTEXT. Advantage is that you don't have to put pTHX to every function's signature and don't have to pass it to every function which works with perl variables. Disadvatage is - decreased performance on threaded perls.
2) Defining PERL_NO_GET_CONTEXT.
Unfortunatelly, as Panda::Install/Panda::XS concept is all about sharing C code, modules that use Panda::XS cannot have different PERL_NO_GET_CONTEXT state. Because in that case, binary incompability between function signatures would occur.
Panda::XS strictly sets PERL_NO_GET_CONTEXT options, so that you have generally to accept pTHX and pass aTHX to all functions that work with perl variables. In this case you'll achieve maximum performance on threaded perls.
However if you don't want to write pTHX/aTHX every time, you can write
dTHX;
at the top of those functions and actually saying revert to behaviour without PERL_NO_GET_CONTEXT (but you still need to pass aTHX to functions which are defined with pTHX, unless called via macros - like perl's API).
For example, you can't receive pTHX in C++ objects destructors, so the options for you are either write dTHX; on the top of destructor or save perl interpreter object in constructor of your object to property named 'my_perl', or use using xs::my_perl global interpreter.
using xs::my_perl
One more problem is static initialization, for example:
# my.h extern SV* myname; # my.cc #include <my.h> SV* myname = newSVpv("john", 0);
This won't work on threaded perls with PERL_NO_GET_CONTEXT because no one defined my_perl. You shouldn't say dTHX; in the global-scoped code. This problem can be solved using using xs::my_perl.
my_perl
This variable holds correct perl interpreter for each thread. Just say using xs::my_perl and you'll get my_perl interpreter for the rest of your code's scope.
Keep in mind that this my_perl variable is get from thread-local storage, like dTHX, so this is actually a convenience tool, reverting to performance as PERL_NO_GET_CONTEXT has not been defined. If you want maximum performance on threaded perls, use pTHX/aTHX everywhere. You could however use mixed style, saying using xs::my_perl and passing pTHX only for performance-critical functions.
dTHX
This variable should not be used on non-threaded perls (this is not a problem as aTHX is empty in that case). However it still exists on non-threaded perls, so that you can say using xs::my_perl without wrapping it into #ifdef PERL_IMPLICIT_CONTEXT.
#ifdef PERL_IMPLICIT_CONTEXT
Another way to achieve maximum perfomance on threaded-perls for C++ objects is to hold my_perl as class variable. In this case you don't have to receive pTHX in every method of your object and don't even have to say dTHX in desctructor.
This helper defines this class member (it defines nothing on non-threaded perls). Example
class MyXSClass { int val; mTHX; MyXSClass (int val) : val(val) { aTHXa(PERL_GET_THX); } // pTHX is not needed anymore in method's definitions. }
Sometimes you have to pass aTHX to constructors of your class members and therefore setting aTHXa in constructor's code is too late.
aTHXa
mTHXa(interpreter) initializes class member my_perl with value interpreter in initializer list. On threaded perls it is my_perl(interpreter),, otherwise empty. Example:
mTHXa(interpreter)
interpreter
my_perl(interpreter),
class MyXSClass { int val; MySVContainer container; mTHX; MyXSClass (int val) : val(val), mTHXa(PERL_GET_THX) container(aTHX) { ... } }
Note that you should not write , after mTHXa. Therefore it cannot be the last on the list.
,
If you don't care about performance on threaded perls then the most convenient solution is using xs::my_perl on the top of the file. In this case you don't have to put pTHX in every function's definition. However you still have to pass aTHX to functions which defined with pTHX.
If module A binary depends on module B and module B updates and becomes binary incompatible, undefined behaviour may happen. To solve this problem, one need to reinstall (rebuild) all modules that depend on module B. To help solving this problem, Panda::Install automatically tracks these depedencies, warns and croaks if any binary depedencies became incompatible and prints the list of modules to reinstall (rebuild).
Pronin Oleg <syber@crazypanda.ru>, Crazy Panda, CP Decision LTD
You may distribute this code under the same terms as Perl itself.
To install Panda::XS, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Panda::XS
CPAN shell
perl -MCPAN -e shell install Panda::XS
For more information on module installation, please visit the detailed CPAN module installation guide.