I’ve had several conversations on Twitter about how it could be a good idea for someone to setup a new community website for PHP components. This website would not host components itself; it would track all of the independently-published PEAR channels, and be the one-stop shop for users to come to whenever they needed to find a component to solve a requirement.

I think this would be a good thing for the PHP ecosystem.

So – requirements! What do you want from such a website? What do you want it to do? How do you want it to work? At this stage, the sky’s the limit. Leave your requirements in the comments, please, and I’ll pull them together into a summary post next weekend.

Once the requirements are stable-ish, I’d like to organise a hackathon to get the first version built and live. If you’re up for that, and want to help make that happen, do ping me on Twitter: @stuherbert.

30 Comments

  1. Lukas says:
    April 9th, 2011 at 10:56 am

    I really like the general direction taken by http://symfony2bundles.org/. Its limited to github.com atm, but it basically picks up all Symfony2 Bundles automatically and then applies a scoring algorithm (the details are of course up for debate): https://github.com/knplabs/symfony2bundles/blob/master/src/Knplabs/Bundle/Symfony2BundlesBundle/Entity/Repo.php#L318

  2. Evert says:
    April 9th, 2011 at 11:24 am

    Just wanted to say: awesome idea! I hope this can take off.

  3. Daniel O'Connor says:
    April 9th, 2011 at 12:01 pm

    So, the current pearweb channels list page/add your channel form is all manual and relies on someone like me applying a patch, and at some later point upgrading pear.php.net to the current version.

    Meanwhile, there’s a bunch of pirium based channels springing up all over the place.

    Some kind of trackback or similar mechanism to automate the registration/publication side of things would be really handy.

    (http://pear.php.net/channels/)

    I think us pear folks would be happy to host whatever you come up with; or link to it at the very least :)

  4. Daniel O'Connor says:
    April 9th, 2011 at 12:08 pm

    Relevant bugs:
    http://pear.php.net/bugs/bug.php?id=13566
    http://pear.php.net/bugs/bug.php?id=13151

    … and the code in that area:
    http://svn.php.net/viewvc/pear/pearweb/trunk/public_html/channels/

  5. Jordi Boggiano says:
    April 9th, 2011 at 12:11 pm

    Just so you know, and to try to avoid a split brain issue, we have a few guys in the Symfony2 community working on such a solution, and while we’d like to use it to track and install Sf2 Bundles and their dependencies, the goal is to keep the tool standalone so that it hopefully gets a broader adoption in the entire PHP community.

    The general idea is to build on top of the PEAR package files, but add a few things that are missing, and especially add support for pulling stuff from GitHub repositories and other target, bypassing the need to setup a PEAR channel entirely.

    I’m sure he won’t like the exposure just yet, but @naderman is working on the dependency solver (which afaik is already pretty far) and then the next step will be to start pulling code, repo can be found at https://github.com/naderman/composer

    Please stop by #symfony-dev on freenode if you’d like to talk to us further, I think it’s important to keep the dialogue open on this one.

  6. Ryan Weaver says:
    April 9th, 2011 at 12:39 pm

    I echo what Jordi said, which is why I really like that you’ve posted a blog asking for input. Per the giant conversation on Twitter yesterday, we’ve gotta aim for united efforts – which I know is your point here.

    I wonder if broadening the approach from just PEAR channels might be a good idea. For example, most libraries (regardless of whether or not that have PEAR packages) are hosted on github. So, if the site were able to show you where a library lives (e.g. github) and the location of its PEAR packages (if it has them), it’ll reach a larger number of libraries. In the Symfony2 ecosphere, a good example of this is http://symfony2bundles.org/.

    Also, check out http://pearfarm.org/ – I’ve just started hearing about it. It might be a good player in all of this, because I HATE making PEAR packages :)

    Cheers!

  7. Josh Johnston says:
    April 9th, 2011 at 2:30 pm

    A PEAR channel proxy would rock. One channel that you can install from and it would transparently proxy the request to the proper backend channel. Then if a channel URL changes or new channels are created, the channel maintainers would only have to update the proxy and all users would get the updates.

  8. Toby says:
    April 9th, 2011 at 5:01 pm

    I did exactly that back when the PEAR channel concept popped up, but the site never went stable and is offline now since ages. However, I still own the domains from back then. If you like the name, I could sponsor the domains. It’s pearadise.net/.org.

  9. Pádraic Brady says:
    April 9th, 2011 at 5:35 pm

    My ideal would be, as Jordi put it, a “channel proxy”. Something that centralises meta data for all channels registered to it, and can proxy installation requests to the correct channel (or through git). Looks like people have worked on large portions of the puzzle – would be amazing if everyone pulled the various strands into one tool and one location.

    This server would maintain a register of packages and meta data. Client installers would request a package from the proxy channel and be redirected to the matching channel for download. I assume PEAR/Pyrus can’t handle this and would need modification for proxies. Updates from member channels could be handled via RSS/Atom and, optionally, Pubsubhubbub to eliminate polling delays.

    The sticky points:

    Should channel registration require a manual form? I’d prefer not – accept a simple pingback from the channel app. Use an Atom feed for metadata, package update detection, and availability checks.
    How to handle channel prefixes? I’d prefer prefixes were ignored. Enforce unique names from member channels so they aren’t needed for proxy purposes.
    PEAR vs Pyrus? I assume Pyrus would be preferential. I honestly don’t know much about it though.

  10. Pas says:
    April 9th, 2011 at 5:42 pm

    Fantastic idea! You should take a look at what Scala is up to regarding packages: http://www.scala-lang.org/gsoc2011 The Scala Index

  11. Stuart Herbert says:
    April 9th, 2011 at 5:54 pm

    Thanks for all the feedback so far. Keep it coming!

  12. Jeremy Coates (@phpcodemonkey} says:
    April 9th, 2011 at 8:00 pm

    Our team started thinking about this since conversations with Stu started in the last 10 days. Our first thoughts were along the “Can’t believe no-one’s done this before!”, and we’re now at the “Yeah, we need to get something going with this” stage. We’re pretty sure the aggregator should have a website for research purposes and a command-line client to the same search results (as most pear/pecl interaction is on the command line) – this allows for a component based approach as suggest in Stu’s recent posts on the topic.

  13. Nils Luxton says:
    April 11th, 2011 at 11:16 am

    I second (or third or fourth) the ideas above about a “channel proxy” – and also agree that @naderman’s composer project would be good to look at / get involved with.

    A single package management tool for PHP, along with a single repository listing all available packages would make life so much easier, both for developers using packages, and developers developing packages.

    Excellent search ability and community rating would be top “wants” from me as far as the website itself is concerned. Automatic tracking of projects would also be great from both the package/website maintainers’ point of view, since it eases the burden on keeping the list up-to-date.

    Very much looking forward to how this progresses!

  14. OnyxRaven says:
    April 11th, 2011 at 4:49 pm

    To keep it lightweight and independent, I’m thinking of a channel aggregator that can just be a meta-channel to a client. PEAR asks for a package and the meta-channel can search for the appropriate independent channel to provide back. Channel servers register with the aggregator and the aggregator adds them to the roster, and periodically indexes the packages. Searches can happen cross-channel.

  15. David Coallier says:
    April 11th, 2011 at 4:59 pm

    This could be VERY easy to automate using SimpleChannelServer and transparent Google Code or Github channel integration.

    Brett wrote a nice post about this a few years back: http://saltybeagle.com/2008/12/using-simplechannelserver-to-manage-a-pear-channel-on-google-code/

    SimpleChannelServer and Pyrus have a lot of potential when it comes to this type of idea. I have tried contacting framework owners before to organise some sort of cohesive collaboration where the general-purpose libraries would go into PEAR2 and anything specific to frameworks we’d direct them to the framework-owners but nothing panned out.

    I don’t think discarding PEAR is a wise choice just yet. I’d be very interested in helping building some sort of channel repository or search-engine and I believe that PEAR could help and play a good role in this.

    For anyone who has inherent issues with PEAR, I’d invite you to contact me directly at davidc [at] php [dot] net and I’d be happy to discuss with you. I’ve noticed that a lot of “issues” most people have are outdated by over 5 years and do not apply to PEAR today.

    Now, the Symfony people have some amazing packages that should be easily findable and distributable. Since they have PEAR channels, it should be easy for anyone that uses PEAR (or that doesn’t want to use PEAR) to find these packages and — be it on PEAR or anywhere else.

    We are at a point in the community where it’s time to work together. I’ve seen too much segregation over the past few years and we need to rally up instead of hiding in our respective frameworks. I understand that some of the frameworks have complex politics issues and I’m sure something can be worked out.

    On behalf of PEAR, I would like to say that this is a really exciting idea and I would personally love to take part in this. I also believe this is exactly what the “PEAR of today” should be made of. We used to be the community, now the community is much wider and we should adapt PEAR to work with external packages. Either way this goes, we are going to support and push this idea forward.

  16. Steven (@SurimZA) says:
    April 12th, 2011 at 1:36 pm

    This is a brilliant idea, and one of which I too have thought of many times before, but just have not had the resources to take it any further than an idea.

    Being a PHP developer myself, I find it to be quite annoying at times looking for a package/component I know must surely be available and inevitably land up writing something up myself.

    I would love to form part of this project. It is something that is most definitely needed, but strangely never done.

  17. Wishing For A PEAR Channel Aggregator? Yes, Please! | Pádraic Brady says:
    April 12th, 2011 at 8:52 pm

    [...] Gathering Requirements For A PEAR Channel Aggregator [...]

  18. Christian Weiske says:
    April 13th, 2011 at 4:21 am

    The PEAR website’s channel list can be grabbed by everyone soon through an XBEL feed:
    http://test.pear.php.net/channels/xbel.php

    Which will soon be available at http://pear.php.net/channels/xbel.php

  19. Jason Lotito says:
    April 13th, 2011 at 12:12 pm

    Have you checked out pearhub.org? It’s a phenomenal service. It’s not everything you are looking for, but it makes contributing a pear installable package dead easy.

  20. Benjamin Dubois says:
    April 13th, 2011 at 7:07 pm

    Hi !

    Love the idea !

    Here are some of my thoughts :

    - Users should be able to install it inside their LAN, for managing their own packages if they want. This could help users to adopt a pear-based packaging and deployment solution, even when their network security prevents webapps to access other webapps through the web. In addition, this would help large companies with several development teams to share code locally (for instance if each team provides it’s own pear channel, with it’s own packages, all aggregated in the solution)

    - Users should be able to create a list of their own apps, and link used packages (from an “all packages” list) to each app. This would help package management on a per-application basis, and simplify the dependency tracking.

    - Most important thing : the app should not be only an aggregator, it should be a deployment tool (Eventually pushing updates to the “client” servers) : I think the goal of PEAR is to help users manage externals components used in their installed apps. Providing a tool that eases package discovery AND deployment could be a real improvement for devs…

    Here are some of my thoughts, hope that helps !

    PS : sorry for my bad english.

    - Benjamin

  21. Wil Moore III says:
    April 14th, 2011 at 7:02 pm

    I also +1 the “channel proxy” idea. BTW, composer seems interesting. This effort needs a google group and/or an IRC channel.

  22. Wil Moore III says:
    April 14th, 2011 at 7:14 pm

    Another thought:

    Has anyone really put any mindshare into thinking about whether an existing (and popular) system should be leveraged or even copied? For example:

    How to use RubyGems.org
    http://rubygems.org/pages/gem_docs

    Building a gem
    http://docs.rubygems.org/read/chapter/5

  23. Padraic Brady says:
    April 14th, 2011 at 9:19 pm

    @Jason – have looked at both Pearfarm and Pearhub. They are both essential elements I think in making it easier for people to distribute packages. Will look forward to Stuart’s requirement list for debate.

    Meanwhile I’ll be blogging a few ideas and code pieces. I would say that we might want to consider keeping something like a specification separate from the implementation. We might end up needing channels to make aggregation easier, and a specification that clearly states what a channel should do will make that easier.

    I know there are problem areas to consider. Where will we get package metadata? How will we handle package name conflicts? It may not simply be about building an aggregation site but also putting in place some best practices a channel should implement (here’s looking at channel hosters and apps) to make this concept work.

    Meanwhile, I’m working on a tangent on some other basic questions about how packaging and installation works. I hope the PEAR folk don’t mind me kicking out code for that while Pyrus is fresh and young.

  24. Gregory Beaver says:
    April 20th, 2011 at 5:21 pm

    As the person who invented the concept and implementation of channels, back when PEAR was just a combination of PEAR and PECL packages awkwardly hosted on one server, I’m very happy to see this problem even existing!

    I encourage everyone who is truly serious about solving them to read the fully complete existing documentation on pyrus at: http://pear.php.net/manual/en/pyrus.php

    Some notes:

    – create a package AND release with 1 command: pyrus make
    – supports TRULY secure channels using openssl-signed package releases
    – easy plugin system used to augment existing functionality (could be used to implement both the github idea AND the aggregator idea without touching a single line of code for pyrus)
    – can already be used to manage a channel directly, simply pyrus install PEAR2_SimpleChannelServer-alpha

    I slaved ENDLESSLY to both design pyrus and to augment PHP itself with the phar extension and some fixes to other areas to enable the incredible new features in pyrus. Please don’t make a horrendously stupid mistake and dismiss or ignore this work.

    Also, be aware that when considering an aggregator, security becomes more important when you hide the source of a package. All package distribution systems for operating systems (such as apt) require explicit entry of the full url of servers for a reason. If channels can be auto-added to an aggregator, and requesting a package name does not require knowing its source, then we have a problem. channels already have a method for specifying mirrors, this can be used to distribute the server that packages are hosted on, and perhaps this could be updated to add aggregation.

    One last note: don’t even bother trying to re-implement dependency resolution. Pyrus does it better than any other program, period. It can handle circular dependencies, including bizarre 3-steps-removed ones, handles stability conflicts, honors existing stability of installed packages, and has unit tests to verify this behavior. It can automatically retrieve new, unknown dependencies from channels it knows about (and from ones it doesn’t know about if you disable the security restriction), as well as static URL-based packages. It handles name conflicts across channels completely as is.

    The reason a channel name is part of the package is because it is too easy to introduce both weird bugs (accidentally installing a package with the same name from the wrong source) or worse, security issues (accidentally installing a package MALICIOUSLY named the same as another you intend to install).

    The best feature that could be introduced for Pyrus would be an offline cache of channels, or a very efficient search/display mechanism so that users can quickly and easily find the package they desire. Both of these are currently missing but I designed it so that they could be easily implemented. Pyrus already caches channel data, the trick would be making it download a local copy and index it.

    Hope this comment is useful. Feel free to email me. I am not actively developing because I am devoting what was my development time to my daughter, but my faculties are fully intact :)

  25. Stuart Herbert says:
    April 27th, 2011 at 7:31 am

    @greg thanks for the comment :)

  26. Daniel O'Connor says:
    April 27th, 2011 at 1:28 pm

    A few minor short open tag hiccups; but http://pear.php.net/channels/xbel.php is now live. I’ve also made it so the channels are editable without a full blown pearweb release; so this feed should reflect anyone who wants to register most of the time. Ping pear-webmaster if we’re slacking off.

    Next item on the wishlist – Fabien to add a “register your pear channel on pear.php.net?” to Pirium.

  27. Alan Pinstein says:
    April 28th, 2011 at 4:40 am

    The channel proxy idea is interesting; I think that it could be something we could support at Pearfarm. I think having a solution to host packages is important and very relevant to aggregation. Pearfarm’s goal is to aggregate packages. It doesn’t really matter to us whether they’re hosted with us necessarily.

    Since we already are a multi-tenant PEAR channel server, it shouldn’t be particularly difficult to proxy a user’s subdomain to another channel.

    Thoughts?

  28. Last Call For Requirements For A PEAR Channel Aggregator « karmrajsinh says:
    May 8th, 2011 at 1:52 pm

    [...] A couple of weeks ago, I posted a call for requirements for a PEAR channel aggregator. [...]

  29. PEAR in July 2011 | PEAR Blog says:
    July 9th, 2011 at 1:37 pm

    [...] We’ve seen an explosion in the number of PEAR channels available. This is coupled with conversations in the community; around how PHP projects can create a robust; diverse ecosystem based on some of [...]

  30. PEAR Channel Aggregator | missjwo says:
    October 27th, 2011 at 8:25 am

    [...] Herbert publishes a blog post gathering [...]