wp-hackers Digest, Vol 109, Issue 25

Shane Thompson

2014-02-20 02:13:15 UTC

Harry,

It seems you've created quite the stir, so let me try and put it in
perspective for you.
If someone walked into your building and started tearing out wall sockets
and checking your electrical wiring then walked up to you and handed you a
sheet of paper saying you failed an electrical inspection, wouldn't you
want to know this person is a qualified electrician?

I agree that we do need something like this, transparent reviews on such
things. However if it isn't making you money, then maybe for now the best
thing to do would be to actually not do these reviews as you lose money on
them. I realise you said that you don't want that kind of advice but it is
what I will give.

I had a quick look at your review of the PODS plugin, as any user would,
without going too far in depth and it looked like a dangerous plugin from
your review. Then I had a look at the github response, and thought "Maybe
it's not that bad". Without having done my own review I can not be certain
any more, is this the kind of response you want? Uncertainty? Who should I
trust? Why should I trust you over the plugin developer? It comes back to
qualifications.

Also, my understanding is you submit such reviews without letting anyone
know. If you wish to improve your current system, you should have some
method of contacting the plugin developers to at least let them know of the
review. If someone was secretly going around telling people not to do
business with you because you don't use recycled plastic in your printers
(or something equally as ridiculous), you'd want to know, surely.

As for your point of requiring payment for a review... Most WordPress
plugins are released open source. As such, the developer makes no money off
it. They invest their time in it for no gains, to find someone has
published a "light-touch" review saying it is dangerous. They will have to
spend their own money, after giving up their own time to have this
re-reviewed (if that's even a word), in the hopes they will have a better
review.

Also, what are you doing to ensure the reviews are not biased to your
programmer? It's very easy to be biased in any circumstance, even in your
profession. When I was in school, my Computer Science teacher did this one
thing to ensure that he was not being biased - instead of writing our names
on our tests, we would write our birth-date. Essentially it came back that
he was not being biased, which is great. But what are you and your
organisation doing to ensure there's no bias in these reviews? Google does
something with their code where it must be submitted for review, and
reviewed by 2 people before it is committed to the code-base. Maybe
something similar could work for you?

Having said all that... If you could address all these issues and make your
reviews transparent, as others have said, and fix these few other issues,
this could be quite a handy resource. However if you are unable to fix this
due to it not being commercially viable to do so, it might be best to
"suspend" the service until such time you can afford to fix it.

If your car is not road-worthy, do you keep driving it until you can afford
to fix it? No, because it is dangerous to do so. So why would you do it
with a service like this?

My advice would be:
1. Set up a standard for these reviews. It needs to be open to criticism.
2. Make it clear that such reviews are simply advisory and are not to be
taken as anything otherwise.
3. Your guys need some credibility. Someone said third-party
certification...
4. There needs to be a way for developers to get back to you, and you
should contact the developers to let them know of the review.
5. Your reviews also should be open to comments, perhaps others could share
their opinions. This would obviously have to be moderated (refer to Chris's
comment on mud fights).
6. You also need to ensure the reviews are not biased.

I realise that doing something out of your own pocket is hard, and quite
respectable when it is for the benefit of others. However unless these can
be fixed you might actually be doing the opposite of what you intend.

Best Regards.

Send wp-hackers mailing list submissions to
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.automattic.com/mailman/listinfo/wp-hackers
or, via email, send a message with subject or body 'help' to
You can reach the person managing the list at
When replying, please edit your Subject line so it is more specific
than "Re: Contents of wp-hackers digest..."
1. Re: WordPress plugin inspections (Harry Metcalfe)
2. Re: WordPress plugin inspections (Eric Hendrix)
3. Re: WordPress plugin inspections (Madalin Ignisca)
4. Re: WordPress plugin inspections (Harry Metcalfe)
5. Re: WordPress plugin inspections (Madalin Ignisca)
6. Re: Fwd: [GSoC - 2014] Introducing Myself (Ian Dunn)
7. Re: WordPress plugin inspections (Harry Metcalfe)
8. Re: WordPress plugin inspections (Harry Metcalfe)
----------------------------------------------------------------------
Message: 1
Date: Wed, 19 Feb 2014 22:22:38 +0000
Subject: Re: [wp-hackers] WordPress plugin inspections
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

snip snip
Does the end user really care how the code is written?
The grade depends on the expertise of the testers. What makes them
qualified to give this grade? Do they have a PHP certification, what's
their background?
The lack of good style must materially reduce the tester's ability to
understand what the code is doing, thereby indicating that the lack of
good style has reduced code readability and maintainability.

This isn't about aesthetics - code that is written in such a way that it
is very difficult to follow is also harder to maintain. It's more likely
to contain bugs, some of which may be vulnerabilities. And it's much
easier to make mistakes when editing it after you haven't looked at it
for a while. It's also evidence that the developer may be inexperienced.
These are all important factors. That said, I can't imagine that a
plugin would fail an inspection on this criterion alone.
The inspections are carried out by experienced developers. I can
appreciate that that might not be clear at the moment. I'm not sure how
we'd go about reassuring people on that front, though: what would you
consider to be good evidence that we're knowledgeable?
Harry
------------------------------
Message: 2
Date: Wed, 19 Feb 2014 14:25:44 -0800 (PST)
Subject: Re: [wp-hackers] WordPress plugin inspections
Content-Type: text/plain; charset=utf-8
Certifications.??
Eric A. Hendrix
910-644-8940

This isn't about aesthetics - code that is written in such a way that it
is very difficult to follow is also harder to maintain. It's more likely
to contain bugs, some of which may be vulnerabilities. And it's much
easier to make mistakes when editing it after you haven't looked at it
for a while. It's also evidence that the developer may be inexperienced.
These are all important factors. That said, I can't imagine that a
plugin would fail an inspection on this criterion alone.
The inspections are carried out by experienced developers. I can
appreciate that that might not be clear at the moment. I'm not sure how
we'd go about reassuring people on that front, though: what would you
consider to be good evidence that we're knowledgeable?
Harry
_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers

------------------------------
Message: 3
Date: Thu, 20 Feb 2014 00:27:13 +0200
Subject: Re: [wp-hackers] WordPress plugin inspections
<CAOztgazyCNP_AWYYF+JmnkWCqHUswDNC8o2=
Content-Type: text/plain; charset=UTF-8
Certification of PHP from a trusted 3rd party source like Zend or similar
company.

Certifications. ?
Eric A. Hendrix
910-644-8940

This isn't about aesthetics - code that is written in such a way that

is very difficult to follow is also harder to maintain. It's more

likely

to contain bugs, some of which may be vulnerabilities. And it's much
easier to make mistakes when editing it after you haven't looked at it
for a while. It's also evidence that the developer may be

inexperienced.

These are all important factors. That said, I can't imagine that a
plugin would fail an inspection on this criterion alone.
The inspections are carried out by experienced developers. I can
appreciate that that might not be clear at the moment. I'm not sure how
we'd go about reassuring people on that front, though: what would you
consider to be good evidence that we're knowledgeable?
Harry
_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers

_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers

--
*Madalin Ignisca*
*web developer*
http://imadalin.ro/
------------------------------
Message: 4
Date: Wed, 19 Feb 2014 22:30:12 +0000
Subject: Re: [wp-hackers] WordPress plugin inspections
Content-Type: text/plain; charset=UTF-8; format=flowed
Hi Eric, Madalin,
That seems reasonable. For the moment - since this is not a service that
makes us any money at all - I think that it's probably not a practical
option. But I will keep it in mind.
You might perhaps draw some comfort from the advisories section. All of
these specific vulnerabilities have been identified by the same testers
that carry out inspections, have been responsibly disclosed and fixed by
the relevant developers.
Harry

Certifications. ?
Eric A. Hendrix
910-644-8940

This isn't about aesthetics - code that is written in such a way that it
is very difficult to follow is also harder to maintain. It's more likely
to contain bugs, some of which may be vulnerabilities. And it's much
easier to make mistakes when editing it after you haven't looked at it
for a while. It's also evidence that the developer may be inexperienced.
These are all important factors. That said, I can't imagine that a
plugin would fail an inspection on this criterion alone.
The inspections are carried out by experienced developers. I can
appreciate that that might not be clear at the moment. I'm not sure how
we'd go about reassuring people on that front, though: what would you
consider to be good evidence that we're knowledgeable?
Harry
_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers

_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers

------------------------------
Message: 5
Date: Thu, 20 Feb 2014 01:02:25 +0200
Subject: Re: [wp-hackers] WordPress plugin inspections
<
Content-Type: text/plain; charset=UTF-8
Hi Harry,
I agree with your idea with the reviews on dxw.com, but as Eric mentioned,
you need some certifications to become an authority and trusted in this.
If not, this will just cause just a dispute and fight on each side.
My personal opinion, I would not trust 100% your reviews as you "green"
some plugins I'd run away from and "red/yellow" a few that really need a
more relevant review, but you have some good points on a few "red" labeled.
Mentioning here on almost all your responses about this service doesn't
makes you money it's pointless, you should be proud that you want to
contribute to the WordPress community and stop complaining about money. If
you want only money, then I suggest you review more on "premium" stuff, as
WordPress.org has a team of members that do reviews and
approve/disapproving plugins and themes and in WordPress.org case we should
have a really nice chat about how we can improve this service so plugins
and themes that would not respect all standards we vote for should be
excluded until corrected as should.

Hi Eric, Madalin,
That seems reasonable. For the moment - since this is not a service that
makes us any money at all - I think that it's probably not a practical
option. But I will keep it in mind.
You might perhaps draw some comfort from the advisories section. All of
these specific vulnerabilities have been identified by the same testers
that carry out inspections, have been responsibly disclosed and fixed by
the relevant developers.
Harry

Certifications. ?
Eric A. Hendrix
910-644-8940

This isn't about aesthetics - code that is written in such a way that

is very difficult to follow is also harder to maintain. It's more

likely

to contain bugs, some of which may be vulnerabilities. And it's much
easier to make mistakes when editing it after you haven't looked at it
for a while. It's also evidence that the developer may be

inexperienced.

_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers

--
*Madalin Ignisca*
*web developer*
http://imadalin.ro/
------------------------------
Message: 6
Date: Wed, 19 Feb 2014 15:40:37 -0800
Subject: Re: [wp-hackers] Fwd: [GSoC - 2014] Introducing Myself
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

1. A *Settings Page* where you will design the Form through a user

friendly

interface like what we use while designing forms using Google Forms.

(*Please

suggest if this will be possible to make in the Required Duration* ).

That's the basic idea, yeah. It would actually be a custom post rather
than a Settings page, though.

2. The Settings page UI, thus, will be used to make new *Forms with an
unique ID*.
3. The Plugin will have a function to generate *a short code* which will
have *arguments like the Form ID* etc.
4. The user can just go to the *'page/post'* that he wants to add the
survey and *use the Short Code with the right ID *as an argument and get
the survey running. !

Yeah, that's all correct. There'll also be some features specific to
WordCamp.org, like supporting pre-defined base forms that organizers can
customize. For instance, a "Call for Speakers" form that potential
WordCamp speakers can fill out. It would have some standard fields like
Name, E-mail Address, WordPress.org username, etc, but then the
organizers could also add their custom questions to the form.
Another feature we're thinking about is having the results of certain
forms automatically populate some of our other custom post types. For
example, when someone fills out the Call for Speakers form, it could
automatically create a drafted Speaker post, so that the organizers
don't have to copy/paste the data between the two.
------------------------------
Message: 7
Date: Wed, 19 Feb 2014 23:45:06 +0000
Subject: Re: [wp-hackers] WordPress plugin inspections
Content-Type: text/plain; charset=UTF-8; format=flowed

Hi Harry,
I agree with your idea with the reviews on dxw.com, but as Eric

mentioned,

you need some certifications to become an authority and trusted in this.
If not, this will just cause just a dispute and fight on each side.

That's fair comment. I suppose we'll just have to see. Of course, people
are free not to use the site!

My personal opinion, I would not trust 100% your reviews as you "green"
some plugins I'd run away from and "red/yellow" a few that really need a
more relevant review, but you have some good points on a few "red"

labeled.
Good. That's exactly how it's supposed to work!

Mentioning here on almost all your responses about this service doesn't
makes you money it's pointless, you should be proud that you want to
contribute to the WordPress community and stop complaining about money.

Sorry if I've come across as complaining. I'm not, at all. We are doing
this in order to contribute to the community - if we didn't care, we
wouldn't have bothered. I hope the site may make some money one day but
that is not it's main motivation.

If you want only money, then I suggest you review more on "premium"

stuff, as

WordPress.org has a team of members that do reviews and
approve/disapproving plugins and themes and in WordPress.org case we

should

have a really nice chat about how we can improve this service so plugins
and themes that would not respect all standards we vote for should be
excluded until corrected as should.

I'm all for that. I think there's room for both! And up for a nice chat
any time :)
Harry
------------------------------
Message: 8
Date: Wed, 19 Feb 2014 23:50:10 +0000
Subject: Re: [wp-hackers] WordPress plugin inspections
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hi Josh,
There's a good thread going on Github.
H

Jamie-
The ability to easily do an independent security review of open source
software, is one of the strengths of the open source model. But

publishing

vague results, and not contacting the developer, and/ or

help

the developer, the community or the users. If anyone can identify a
address it, like any other responsible developer would.
Take care,
Josh

I had the exact opposite reaction to Chris Williams. Literally a week

ago

I was talking to someone about the need for more rigorous evaluation of
plugins. I find that I now use only a small handful of plugins that I

have

extensive experience with because of the lack of any quality standard.
If that sounds a bit harsh, I'd suggest enabling DEBUG and mysql slow
query (at something like 1 second) and then test out various plugins.

And

that's just the blatantly obvious stuff. I won't point fingers, but I
recently had issues with one pretty popular plugin and when I went into

the

code to poke around I found that it is fundamentally flawed in the

design

-- so much so that I rewrote it and will be sending the author the new

code

and explanation.
I understand that a cursory review is subjective and prone to
misstatements, but it's at least a step in the right direction. Perhaps

the

next step would be for Harry to formalize some kind of process for
responding to / contesting reviews and to encourage community

involvement

(maybe via this list) to "review the reviews" if you will. I'd be happy

get involved in a process like that if the end result were a base of
plugins that had been scrutinized by some of the WP brains on this list.
And if, at the end of the day, he harnesses that power to help build a
business, I don't see anything wrong with that either. I think 99% of us
are using WP to make money and it seems to me like he's identified a

clear

need and at least attempted to address it -- which is pretty much the

story

of every successful business.
Jamie Currie
Founder / CEO
wunderdojo
wunderdojo.com
tel: 949-734-0758
1840 Park Newport, #409
Newport Beach, CA 92660
Master web & app developers
------ Original Message ------
Sent: 2/19/2014 12:17:17 PM
Subject: Re: [wp-hackers] WordPress plugin inspections

I certainly can't speak for others, but I would venture to say that

your

business model is evil at best. You do fly-by character assassination
(oops, I mean "light-touch inspections"), based on personal bias ("this
plugin is large"), and then broadly publish the results as if they are
somehow authoritative. Worse yet, you then hold plugin developers at
ransom for changing the review: "If you would like to commission us to
inspect or review the latest version, please contact us."
How this is of value to anyone, and how you sleep at night with this
specious business model, is completely beyond me.
Hello list,

We write and publish light-touch inspections of WordPress plugins that
we do for our clients. They are just a guide - we conduct some basic
checks, not a thorough review.
Would plugins which fail this inspection be of general interest to the
list and therefore worth posting? Is the list also interested in
vulnerability advisories, or do people tend to get those elsewhere?

https://security.dxw.com/plugins/pods-custom-content-types-and-fields/

Grateful for a steer...
Harry
--
Harry Metcalfe
07790 559 876
@harrym
_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers

_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers

------------------------------
Subject: Digest Footer
_______________________________________________
wp-hackers mailing list
http://lists.automattic.com/mailman/listinfo/wp-hackers
------------------------------
End of wp-hackers Digest, Vol 109, Issue 25
*******************************************

--
*Kind Regards*
Shane Thompson

T - 08 9350 9392
F - 08 9356 6168
E - ***@webwizards.com.au
W - www.webwizards.com.au